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AUTHOR’S PREFACE 


The numerical solution of the problems of mathematical 
physics is most frequently connected with the numerical solution of 
basic problems of linear algebra—that of solving a system of linear 
equations, and that of the computation of the proper numbers of a 
matrix. The present book is an endeavor at systematizing the most 
important numerical methods of linear algebra—classical ones and 
those developed quite recently as well. 

The author docs not pretend to an exhaustive completeness, hav- 
ing included an exposition of those methods only that have already 
been tested in practice. In the exposition the author has not strived 
for an irreproachable rigor, and has not analysed all conceivable 
cases and sub-cases arising in the application of this or that method, 
having limited herself to the most typical and practical important 
cases. 

The book consists of three chapters. In the first chapter is given 
the material from linear algebra that is indispensable to what fol- 
lows. The second chapter is devoted to the numerical solution of 
systems of linear equations and parallel questions. Lastly, the third 
chapter contains a description of numerical methods of computing 
the proper numbers and proper vectors of a matrix. 

For the interest manifested in the manuscript, and for a number 
of valuable suggestions, I express my sincere thanks to A. L. Brudno 


and G. P. Akilov. 
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TRANSLATOR’S NOTE 


But for the initiative of Dr. George E. Forsythe, this trans- 
lation would not have been written; it is thanks to his awareness 
and appreciation of this work in the original, as well as to his support 
of the translating, that this English-language version now appears. 
And to Mrs, Faddeeva, the author, go my respects for a real com- 
putor’s guide-book, with a nice sense of both theory and practice, 
and a presentation no less nice—a book such as I wish I had had. 

Grateful acknowledgments are duc to my father, L. Halsey 
Benster, for contributions beyond a mechanical engineer’s line of 
duty (including proofreading in particular), and to my wife Ada, 
for more than can be expressed. June Wolfenbarger’s fastidious 
typing helped get the manuscript off the ground. 

I have added a few notes [those in brackets], replaced many of 
the Russian references with more accessible ones, and re-computed 
all of the principal tables, which I hope will thus be especially 
reliable and useful guides. 

Curtis D. BENSTER 
Ophir, Colorado 
1958 
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COMPUTATIONAL 
METHODS OF 
LINEAR ALGEBRA 


CHAPTER I 


BASIC MATERIAL FROM LINEAR 
ALGEBRA 


This chapter will be of an auxiliary character. Without 
detailed proofs, it will impart material from linear algebra that 
will be indispensable to an understanding of the following chapters. 


§ 1. MATRICES 


1. An aggregate of numbers—which are, generally speaking, 
complex—arranged in the form of a rectangular table, is called a 
rectangular matrix. This array will have m rows and n columns, and 
may be set forth in the form: 


411 41g N, 
Qa, Gown ... ax 

(1) A= =|, 
Amt Aye +++ Gan 


the first subscript, then, designating the row, the second designating 
the column, in which the element in point is located. 
This may be abbreviated to the form: 


A=[a,] (i=1,2,...,m;j=1,2,...,2). 


Two matrices are equal if their corresponding elements are equal. 
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Matrices composed of a single row are called simply rows (or, as 
they will be approached later, row vectors). Matrices composed of 
a single column are called columns (or column vectors). 

If the number of rows of a matrix equals the number, 2, of 
columns, it is called square, and of the nth order. 

Among square matrices, an important role is played by diagonal 
matrices, i.e., matrices of which only the elements along the prin- 
cipal (leading) diagonal are different from zero: 


Qa 0 eee 0 
0 a ... 0 

(2) ne tegen ak. Seay 
0 0 a, 


If all the numbers a; of such a matrix are equal to each other, the 
matrix is said to be scalar: 


a O... 0 
(3) Oa ... Of = Fay 
0 0 a 


and, if a=1, the matrix is said to be the unit matrix: 


10... 0 
O11... 0 

(4) =I. 
00... 1 


Lastly, a matrix all of whose elements are equal to zero is called a 
null matrix, or zero matrix. We shall designate it by the symbol 0. 

The determinant whose elements are the elements of a square 
matrix (without disarrangement) is said to be the determinant of that 
matrix, and we write the determinant of the matrix A as |], or often 
as d(A). 

2. Multiplication of a matrix by a number. The addition of matrices. 
A matrix whose elements are obtained by multiplying all the ele- 
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ments of the matrix 41 by a number a is called the product of the 
number @ and the matrix A: 

Gay, Gayo «1. Ay, 


(5) eh = Man, Adon ... Gan, 


4,,, OA, 


A matrix C whose elements are the sums of the corresponding ele- 
ments of A and B, matrices having like numbers of rows and columns, 
is called the swmn of A and B: 


24,45), ayatbyn ... ay,+5), 
(6) A+B= Ga,t+b2, dogtby, ... dag t bay 
Gn + Bn) 4,9 + 642 eee Gian + Oman 


The operations introduced above have the following properties, 
as will be readily seen: 


A+(B+C) = (A+B) 4+C. 
A+B = BLA. 
A+0 =A. 


(e+f)A =aA+fa. 
a(A+B) = ad+eaB. 


OP ee ae at ee 


Here A, B, and C are matrices; « and # are numbers—generally 
speaking, complex. 

3. The multiplication of matrices. Multiplication of the matrices A 
and B is defined only on the assumption that the number of columns 
of matrix A equals the number of rows of matrix B. On this 
assumption, the elements of the product, C=AB, are defined in the 
following manner: the element in the ith row and the jth column 
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of the matrix C is equal to the sum of the products of the elements 
of the ith row of the matrix A by the corresponding elements of 
the jth column of matrix B.! Thus: 


4yy @yq wee Ay by, bye eee by, 


(7) AB Go, G22 +++ Gay | | bar bay .-- bay 


where 
n 
Cig = G30; +Gjobos+ +++ +4 jqdg, = = 444; 


(8) 
(i=1,2,..., msj=1,2,..., p). 


It is to be noted that the product of two rectangular matrices is 
again a rectangular matrix, the number of rows of which is equal to 
the number of rows of the first matrix, and the number of columns 


of which is equal to the number of columns of the second matrix: 
mxn nxp mxp 2 . 
-B=C. So, for instance, the product of a square matrix and 


a matrix composed of one column is a matrix of one column. 

The commutative law for multiplication does not, generally 
speaking, hold. We shall make a few observations on this subject, 
however. The matrices AB and BA make sense simultaneously only 
if the number of rows of the first matrix is equal to the number of 
columns of the second, and the number of columns of the first is 
equal to the number of rows of the second. Given the fulfillment 
of these conditions, the matrices AB and BA will both be square, but 
of different orders, unless A and B be square. ‘Thus even to put the 

1 [As to the motive for this apparently arbitrary definition, sce, e.g., AITKEN, 


A. C., {2}, § 3. Abundant numerical illustrations will be found in Frazer, R. A., 
Duncan, W. J., anp Cotrar, A. R.,[{1}, § 1.4.] 
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question of the equality of the matrices AB and BA makes sense only 
for square matrices. But even in this case, generally speaking, 
ABABA. 

In particular cases multiplication may be commutative, and in 
such cases the matrices arc said to commute. Thus, for example, 
scalar matrices commute with any square matrix of the same order, 
for 


a O ... 0 1 Aq «ee Ay 
Ow... O Ga, Gon .-- Gay 
(9) 
0 0 a Gy) Aya eee Any 
ay, @o .- Qin «@ 0 ... 0 
Qa, Gag --- Aa, 0 « 0 
Gy Ayn ses Any 00. a 


a,j; GOA,o ..- AAp, 


Hence follows the special role of the unit matrix in the multiplica- 
tion of matrices, to wit: amongst all square matrices of the same 
order, the unit matrix plays the same role as the number one docs 
among numbers. Indeed, 


Al=JA =A, 


It can be shown that the multiplication of matrices is associative, 
viz.: if AB and (AB)C make sense, so also do BC and A(BC), and 


1. A(BC) = (AB)C. 
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The matrix product has also these properties: 


2. a(AB) = («A)B = A(aB); . 
3. (A+B)C = AC+BC; 
4. C(A+B) = CA+CB, 


where A, B, C are matrices, @ a number. 
Let us interchange rows and columns in the matrix 


411 ByQ ee Ay 


Cr 


a; Gay ant 
Q@yo Ang «ee Ang 

(10) Ao = AT = [4;;]’ = ie = [a;;]. 
Bin 42 Gan 


The following rule (the reversal rule) for a transposed product 
should be noted: 


(AB)’ = B’A’ 

In proof of this, note that the clement of the ith row and jth 
column of the matrix (AB)’ is cqual to the element of the jth row 
and ith column of the matrix AB, for this is merely the interchange 
of row with column, i.e., transposition; and that is equal to 


(11) 4;1b1; +4jnbo; + ++ + +4 jyb,;- 


The last expression is obviously equal to the sum of the products of 
the elements of the ith row of the matrix B’ and the corresponding 
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elements of the jth column of the matrix 4’, i.c., is equal to the 
general element of the matrix B’A’. 

In conclusion we shall remark that the determinant of a product 
of square matrices is equal to the product of the determinants of the 
multiplied matrices: |AB| =|A| |B], which result is taken from deter- 
minant theory. 

4. The partitioning of matrices. The handling of matrices of high 
orders requires, as a rule, a large number of operations. It is there- 
fore often convenient to reduce a computation involving matrices 
of high orders to computations upon matrices of lower orders. Such 
a reduction can be effected by partitioning the given matrices: each 
matrix may be conceived as composed of several matrices of lower 
orders, and this subdivision may be carried through in many ways, 
for example: 


411 42 433 414 a1) > 42 41g 41g 
Qo aay ayy ay, HRseHe Hse e se ike guieelte sees aewae . 
43; 432 433 434 Qa, tay Ang Avg 
431 > 4@gz 433 34 
41 42 3 aig a4, 
a2) Qow : any Ans 
43, 4yp 2 gg aay 


The matrices into which the given matrix is partitioned are called 
its submatrices, or cells. In such a partition the horizontal and ver- 


tical subdividing lines are of course supposed to be carried across the 
whole matrix. 


We shall not concern ourselves with the general case of the parti- 
tioning of a matrix,! but shall here consider only a partition of 
square matrices in which the diagonal submatrices are square. 

The basic operations on partitioned matrices whose diagonal 
matrices are of identical orders is connected in a quite natural way 


1 [Sce, e.g., Arrken, A. C., [2], §11.) 
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with operations upon the submatrices themselves. To wit, if we 
have 


An Ais Ay 
qe As, Anes Ay, 
Au Aye Aig 
and 
By By By 
B= By, Bay Boy ; 
By By Bu 


where A;; and B,; are square matrices of the same order, then 
AntBy AwtBy .-. AgtBy 
Agi + Ba, Agot Bog... Ay tBry 


(12) A+Be= 
AytBy AwtBie --- dutBu 

and 

Cy Cr Cy 

Cy Ca eee C. 
(13) AB=| *) al fe 

Cu Cy Cy 
where 


C;; = Ay By + Aj Bo + see +Ai,Bi; (Gi,j= l, weeny k). 


We shall not stop for a proof of the last formula, but will note here 
only that the matrices A;, and 8; can indeed be multiplied, since 
the number of columns of matrix 4;; equals the number of rows of 
the matrix By; 

Formulas (12) and (13) show that operations with matrices parti- 
tioned in the manner indicated are to be performed just as if in 
place of cach submatrix there was a number. 
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An important special case of a partitioned matrix is the bordered 
matrix. Having a square matrix «1,_, of order n—1: 


a1 a2 see By a-l 


ae} aa see Ga) gol 


G,-1, 1 a,-1, woes And, nl 


we form a square matrix of the nth order, A,, by appending to the 
matrix A,_, a row: U,1=(@y1,--+5 4, 9-1), @ column: 4,_1 
= (44,) ---4,-1, ,), and a number a,,: 


a), 
Anat Q2n ! Fy 
: “Fart —1 
(14) A, = . - (i le 
Uae Gn 
ay_1, n 
Ps 4,, n-1 Fan 


We shall say that the matrix 4, has been obtained by bordering the 
matrix «1,_,. The matrix A, is naturally partitionable. 

Operations upon a bordered matrix are performed in accordance 
with the general rules for operations upon partitioned matrices. 


Letting 
Mi ou Py 
A= ; B= 
|: A [. b 


be two bordered matrices of order a, the meaning of M, v, u, a, and 
P, x, y, 6, being those of the definition, the following statements are 


valid: 
( | aM au | 
aA = : 
av aa 
Mf+P : 
(15) A+B = : i ails 
u+x at+b 


< 
MP+ux My+ub 
AB = . 
vP+ax vy +ab | 
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Here MP and ux are matrices of the (n—1)th order; My, ud are 
columns composed of n—1 elements; vP and ax are analogous rows; 
and, lastly, vy +d is a number. 

5. Quasi-diagonal matrices. Let us consider still another particular 
case of partitioned matrices, namely the matrices called quasi- 
diagonal. These are square matrices along whose leading diagonals 
are arrayed square submatrices, the remainder of the elements being 
zero. An example would be the seventh-order quasi-diagonal 
matrix: 


4, a : 0 O O : 0 0 


401 40 0 0 0 0 0 
0 0 bi bie big 2 0 
0 0 boy boo bog 0 0 
0 0 bai Bas. Bag 0. 10 
0 860 0 Oo 0 C1 12 
0 0 0 0 0 Cy1 fgg 


The cells of this matrix are obviously 
Ate i yy 
G21 Ae 


B= | ber bee bea | 5 c= | ] 


f21 Ce 
bai b32 bsg 


and the six null submatrices. 

If two quasi-diagonal matrices are of like structure, the product of 
such matrices will also be a quasi-diagonal matrix of the same struc- 
ture, the diagonal cells of which equal the products of the corre- 
sponding cells of the factor matrices. 

The determinant of a quasi-diagonal matrix is equal to the product 
of the determinants of the diagonal cells, on the strength of a notable 
theorem by Laplace.} 


1 [See, e.g., Atrnen, A. C., [2], § 33. 
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6. The inverse and the adjoint matrices. A square matrix A=[aq;;] is 
said to be non-singular if its determinant is not equal to zero; in the 
contrary case it is of course singular. 

The important concept of an inverse matrix is now introduced. 
A matrix B is called the inverse (or reciprocal) of the matrix A if 


(16) AB = |. 


We shall show that the necessary and sufficient condition for the 
existence of the inverse matrix is the non-singularity of the matrix A. 

The necessity follows at once from the theorem concerning the 
determinant of a matrix product, for if AB=/, |A| |B|=1 and con- 
sequently |A] #0. 

Assume now that |4|40. In order to construct the inverse 
matrix we must give preliminary consideration to the adjoint (or 
adjugate) matrix, i.c., the matrix 


Ayy Ag... Aga 


(17) C= 


Here A;; is the algebraic complement (cofactor) of the element a;; in 
the determinant of the matrix A, i.e., is the signed minor deter- 
minant of the element a;;. Note that it is here placed in transposed 
position. 

We shall show that the adjoint matrix has the following property: 


(18) AC = |All 


In demonstration, reckoning the general clement of the matrix AC 
by the rule for matrix multiplication, we find it to equal 


4,14 +4j24 jot +--+ +4;,A jy 


ie., zero for i#y, and |Aj for i=j, on the strength of a familiar 
theorem on the expansion of determinants.! 


1 (See, e.g., Arrxen, A. C., [2], § 21.} 
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The equality 
(18") CA = |All 


is established in like manner. 
The adjoint matrix has meaning for any square matrix A. From 
the equality AC=|A|/ it follows that the matrix 


l 
19 —s 
(19) B [a c 


is, for non-singular A, the sought inverse, for 


l l 
AB = A—C = —AC =I. 
[A] ~~ [ay 


The constructed matrix has also the property 
(20) BA =1, 


which follows from equation (18’). 

We prove, lastly, the uniqueness of the inverse matrix. Assume 
that a matrix X exists such that AX=/. Multiplying this equation 
by B on the left, we have X=B. If it be assumed that YA=/, a 
multiplication on the right by B yields Y= B. 

The matrix inverse to A is denoted by A-!. It is obvious that 
|A~"] =|4]-1. 

We note that the inverse of the product of two matrices also 
displays the reversal rule: 


(21) (A,Aq)-! = AZ! Az, 


since 
A,Ag43'A;! = A,4;! = 7, 


The determination of the inverse matrix is one of the fundamental 
problems of linear algebra. Equation (19) offers the possibility of 
computing the inverse matrix; however, the computation of the 
adjoint matrix is so labor-consuming that the cited equation is of 
importance only in theoretical relationships. Chapter II will be 
specially devoted to this problem of determining the inverse matrix. 
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7. Polynomials in a matrix. We now define the positive integral 
power of a square matrix, putting 
n times 


22 pci acre 
) AA. ...-A = Ap, 


In view of the associative law, how the parentheses in this product , 
are placed makcs no difference, and we therefore omit them. It is 
evident from the definition that 


ArAm = Antm 
(23) { ; 


(An)™ = Anm, 


Hence it follows that powers of the same matrix are commutative. 
We further put, by definition, 


Ao = J, 


An expression of the form 
agdn+a,Ae-l+ .-- +e,/, 


where ao, &),-..,@, are complex numbers, is called a polynomial in 
a matrix, or matrix polynomial. This matrix polynomial may be 
regarded as the result of replacing the variable 4 in an algebraic 
polynomial 


(24) p(A) = apd" +a,J-14+ --- +a, 


by the matrix 4. 

It is important to note that the rules for operation upon matrix 
polynomials do not differ from the rules for operation upon algebraic 
polynomials, viz.: 


given (A) = (A) £x(A) 
w(A) = y(A) x(a), 

then = g(a) = p(A)+7(A) 
w(dA) = y(4) x(4). 


This follows from the commutativity of the powers of a matrix. 


(25) 
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8. The characteristic polynomial. The Cayley-Hamilton theorem. The 
minimum polynomial. The equation 


43;—A aye 41, 
a Goo A... 

(26) 2 22 Qn =0 
ay) 4,9 Qy,—A 


is called the characteristic, or secular, equation of the matrix A=(a;;). 
The left member of this equation, which may be written in the 
abbreviated form |A—4AI|, bears the name characteristic polynomial (or 
characteristic function) of the matrix. Characteristic equations are 
frequently encountered in applied mathematics. 

The direct computation of the characteristic function presents 
considerable technical difficulties. If 


(27) (A) = |A-Al] = (= 1)e[ar—pydet pad «+ —p,), 
then 


1 = 41; +499+ +--+ +4,,, 
(28) 


by = (—1)14I, 


and the remaining coefficients p, are the sums, taken with the sign 
(—1)4-}, of all the principal minors of the determinant of matrix A of 
order k, i.e., of the minors involving the principal diagonal.!| The 
number of such minors equals the number of combinations of 2 
things taken & at a time. 

The roots of the characteristic equation are called the proper 
numbers (or characteristic roots, latent roots, proper values, eigenvalues) of 
the matrix A. From the well-known theorem of Vieta giving 
the connection between the roots of an equation and its coefficients, 
we have 


(29) { AitdAgt +++ +4, = fi = au taget+ --- +4,, 
AyAg ewe An = (-1)*-p, = |A|. 
2 (See, e.g., Armen, A. C., (2], §37.] 
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The quantity p;=a,;+4@22+ +--+ +4,, is called the trace (or spur) 
of the matrix A, and is denoted by tr A. 

Practically convenicnt methods for determining the coefficients 
and roots of the characteristic equation will be elaborated in 
Chapter III, which will be specially devoted to that group of ques- 
tions. For the moment we leave them aside. 

For any square matrix the following remarkable relation, known 
as the Cayley-Hamilton theorem, obtains: if (A) is the characteristic 
polynomial of the matrix A, then g(A) =0, that is, in a sense, the 
matrix is a root of ils own characteristic equation. 

For proof, let us consider the matrix 8, the adjoint of the matrix 
A—2I, Since each cofactor in the determinant |4 —A/| is a poly- 
nomial in / of degree not exceeding n— 1, the adjoint matrix may be 
represented as an algebraic polynomial with matrix coefficients, 
ic., in the form 


B = B,_1+8B, wvdt+ sissy: + Bodr-!, 


where B,_;,..., Bo are certain matrices not dependent on A. On 
the strength of the fundamental property of the adjoint matrix, we 
have 


(Bi +B, A+ +++ +Bodt-!)\(A-Al) = [A—All 
= (—1)9(An—p,dr-1— --- —p, 1. 


This equation is equivalent to the system of cquations 
B,-1A a (2 I)attp 
B,-2A—B,-1 = (=1)8*3p.a41 
BoA-B, = (—I)stlpT 
—Bp = (=I) 
Multiplying these equations on the right by /, A, A2,..., A*-!, As, 


respectively, and adding, we obtain a null matrix on the left side, 
and on the right, 


(30) (— 1)" -p, —pa-1A —py-2l? — +++ +A] = (A). 
Thus (A) =0, which is what was required to be proved. 
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The Cayley-Hamilton relation shows that for a given square 
matrix a polynomial exists for which it isa root. Evidently such a 
polynomial is not unique, for if y(A) has such a property, so has any 
polynomial divisible by y(A). The polynomial of lowest degree 
having the property that the matrix A is a zero of it, is called the 
minimum polynomial of the matrix A. 

We shall prove that the characteristic polynomial is divisible by 
the minimum polynomial. 

Let ¢(A) and r(A) be the quotient and remainder obtaincd upon 
dividing the characteristic polynomial g(A4) by the minimum poly- 
nomial (A): 


GA) = (A) 9(A) +r(A), 


the degree of r(A) being of course less than the degree of p(A). 
Substituting A for 2 in this equation, we have 


r(A) = (A) (A) (A) = 0. 


Thus the matrix A proves to be a “zero” of the polynomial r(A); it 
thence follows that r(2) =0, since otherwise (4) would not be the 
minimum polynomial. Consequently p(A) divides y(A). 

9. Similar matrices. The matrix B is said to be similar to the 
matrix A if a non-singular matrix C exists such that B=C-!AC. 
Matrix B is said to be obtained from matrix 4 by a similarity trans- 
formation. : 

The similarity transformation has the following properties: 


lh C-1A,C+C7 ACH ++. +C-14,C 
= C-!(A, + At bile +A,)C. 


(31) 2. C-1A\C-C-140C +...» C-1A,C = C-1(A,Ag.. AC. 


In particular, (C-!.AC)= = C-!A0C, 
Hence: 


3. f(C“1AC) = C-1 f(A) C for any polynomial f(A). 
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From the last property it follows directly that similar matrices have the 
same minimum function. 

We shall show that similar matrices have also the same characteristic 
Junction. 

We have 


|B—2l| 


|C-1AC—Al| = |C-1AC- AC-1IC| 
|C|-'JA -— AN] |C] = JAA]. 


10. dclementary transformations. tis frequently necessary to effect 
. the following operations upon matrices: 

a) Multiplication of the elements of some row by a number; 

b’) Adding to the elements of some row numbers proportional to 
the clements of some preceding row. 

b’) Adding to the elements of some row numbers proportional to 
the clements of some following row. 

Sometimes such transformations must be made upon the columns. 
Transformations of the type indicated are called elementary trans- 
Sormations of the matrix. 

Any elementary transformation of the rows is equivalent to a 
premultiplication of the matrix (multiplication on the Icft) by a non- 
singular matrix of a special form, as will readily be verified. 
Operation a) is equivalent to a premultiplication of the matrix by 
the matrix 


(32) a 


we 


operation b’) is equivalent to a premultiplication by the matrix 
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(33) ; 


] 
operation b”) is equivalent to a premultiplication by the matrix 


1 


(34) 


l 


Operations a), b’) and b’) are performed on columns by using just 
such elementary matrices! in postmultiplications of the matrix under- 


1 [Note that the elementary matrices may well be regarded as having been 
derived from the unit matrix by just such transformations, a), b’), and b’), as it 
is proposed to make upon the rows/columns of A. Thus, for example, the second 
elementary matrix of the example effects, by premultiplication, the following 
transformation of A: row 3+ row 2, and is itself the result of such an operation 
upon the unit matrix. By postmultiplication it effects: column 2+a column 3, 
and is this transformation of J.] 
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going transformation, (33) now effecting b”) upon columns, and 
(34) effecting b’). 
Examples of row operations: 


l abe ab ¢ 

0 a 0 x y 2] =| ox oy az]; 

00 1 u w “wove ew 

1 0 0 abe a b ¢ 

0 1 0 x y zi x y z ; 
0a il u w utax v+tay w+az 

1 0 0 a ¢ a b ¢ 
Ole x y Z| =|] xtau ytav z+aw 
00 1 u w u v w 


In future work we will often have to perform transformations of types 
a) and b’) upon matrices. The result of several row-transformations 
of this type is equivalent to the premultiplication of the matrix by 
some triangular matrix, i.e., one of the form 


(3 5) Y21 


rr 


Yat Yn2 «++ Yan 


with non-zero diagonal elements y;;._ Indeed, each separate trans- 
formation of form a) or b’) is equivalent to a premultiplication by a 
triangular matrix of the type indicated, and the product of two or 
more triangular matrices of like structure (i.e., both, say, dower 
triangular, as these) is again a like triangular matrix. 

It should be further noted that the result of several column- 
transformations of the form b’) and b’), such that to each, jth, 
column is added a multiple, m;;, of the elements of the ith column 
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(which itself remains unchanged), is equivalent to postmultiplica- 
tion of the given matrix by a matrix of the form: 


V0! of aghast 0 
0: Hise oa te ee oh ae 20. 
(36) # CEE Meck Bok Seley get sg I 


MM; Mg oe ME 5-1 1 M; gt1 ++ Mig 


OOM 6. 2 ie en ah il ed 


11. Decomposition of matrices into the product of two triangular matrices. 
Triangular matrices, that is, matrices of the form 


Cy 0 1. 0 by, bie eae 5, 
2 «1. O 0 boo... do 

(37) ia and a cies | 
€ny Cag ee lay 0 0 ary ban 


(no ¢;;=0, no b;;=0) 


have a number of convenient properties. For instance, the deter- 
minant of a triangular matrix equals the product of the elements of 
the principal diagonal; the product of two triangular matrices of 
like structure is again a triangular matrix of the same structure; a 
non-singular triangular matrix is easily inverted and its inverse is of 
like structure, etc. 

The following theorems are therefore of interest. 


THEOREM. On condition that the leading submatrices of the matrix 


41) 43g «+s Gy 


G1 Qgg ... @ 
A 22 2n 


are non-singular, i.¢., that 


ay, # 0, 
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A may be represented as the product of a lower triangular matrix and an 
upper triangular matrix. 

The proof will be carried through by the method of mathematical 
induction, 

For n= 1, the statement is obvious: (@);) =(411)(¢11), and one of 
the factors may be taken arbitrarily. Let the theorem be true for a 
matrix of the (2—1)th order. We shall show it to be true for 
a matrix of the ath order. 

Partition the matrix 4 into a bordered matrix: 


ay MN, 41, 
Ae as, an, = Ani 
an-1, a 
41 Ann G1 An2 Gy, n-1 Qng 


= ee u F 
“a Daan}? 


we shall seck a decomposition A =CB of the matrix A into the pro- 
duct of two matrices B and C of the required forms, first having 
partitioned these matrices into bordered form like that of A: 


e=[% 0 : a [2° y |. 
Xo. Cia 0 4,, 


By the rule for multiplication of partitioned matrices: 


C1 07 Bi y 
CB = n n 
| *« Can | | 0 ban 
= ies aa Gy | =A 
LE xB,-) XY + CynP nn : 
whence we have 


C,- 18,-1 _ A, 
Now such triangular matrices, C,_, and B,_, exist, by the induction 


hypothesis. Furthermore, from the assumption that [4,_,|#0, it 
follows that |C,_|#0 and |B,_,|#0. 
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Now « and y are found by the formulas 
¥= Cyl, += vBrs,, 


wherewith they are determined uniquely in terms of u and v. 
Thus it only remains for us to determine the diagonal elements 
¢,, and 6,, from the equation 


CanPnn = ay, — XY. 


The last equation shows that one of the diagonal elements may be 
taken arbitrarily. 

Thus the decomposition of a matrix into the product of two triangular 
matrices, lower and upper, will be unique if we prescribe values for the 
diagonal elements of one of the triangular matrices. 

It is convenient to consider, for example, that 6;,=1,1=1,..., 2. 
Then 

fon = Gnn— XY 


and accordingly matrix C will be uniquely determined. 
12, Matrix notation for a system of linear equations. Let us consider 
the system of n linear equations in n unknowns: 


4X) +4yo%e+ ++ > +AiyX, = by 


Go1X1 +Aeox%o+ +++ +ao,x, = 8 
(38) 21%1 FageXe Qn*a 2 


41%, +4,0%o+ ++ +> +4,,%, = 6,. 


Utilizing the definition of matrix multiplication, the system may be 
written as a single equation in matrix notation: 


44, 4p 6 Oty *1 by 

a a feed x b 
(38") 21 a9 Qn e = 2 

4, 4,2 ann Xn b, 


or simply as 
(38") Ax = b, 


A signifying the matrix of the coefficients of the system, 4 the column 
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of constant terms, and x the column whose elements are the un- 
knowns. 

If the matrix of the system, A, is non-singular, we obtain at once 
the solution of system (38) by premultiplying (38") by A-!: 


l 
39 x = A-1b = — Bb, 
B being the adjoint matrix of A. 
We shall show that the last formula is the matrix notation for the 
familiar Cramer’s Rule: 


_ IA 

(40) 4, = lay’ 

where 4, is the matrix that is obtained from A by replacing the ele- 
ments a,; of the ith column by the components 6, of 6. 

Indeed, the matrix equation (39) is equivalent to the 2 equations 


(40a) x; = cy ot Aniby (i=l,..., 2). 


Since the 4l,; are the cofactors of the element a,; in the determinant 
of the matrix A, we obviously have 


(40b) Ay jb, + Agibet +++ +Anid, = Ail, 


which proves our statement. 


§ 2. n-DIMENSIONAL VECTOR SPACE 


In what is to follow an important role will be played by the 
so-called n-dimensional vector space R,. A point X of such a space is 
an aggregate of n numbers, as a rule complex, arrayed in a definite 
order: 


(1) X = (x, Xa)... Xq)s 


X is also called an n-dimensional vector. The numbers x), x2)... 5 X, 
are called the components of the vector. The number z is called the 
dimension of the space. 

Two vectors are said to be equal only if their corresponding com- 


ponents are equal. Fundamental operations on vectors are defined 


24 Basic Material from Linear Algebra 


as follows: If X= (x), xo,...,*,) and Y=(y;, yo,..., y,) are two 
n-dimensional vectors and @ is an arbitrary complex number, we 
then put, by definition, 


(2) { X+V = (x1 +91, to+Yo) +++ > Xat+9n) 
aX = (ax,, AXq,..., a%,). 
The addition of vectors satisfies the commutative and associative 
laws: 
X+Y=YV4+X 
(X+Y)4+Z = X+(Y¥+Z). 


The addition of vectors is connected with multiplication by numbers 
by the distributive laws 


a(X+Y) = aX+aY 


(3) (a+b)X = aX+bX. 


The validity of all these laws follows directly from the definition 
of the operations. 

For vectors of an n-dimensional space a scalar product is intro- 
duced in accordance with the formula 


(4) KY) = 3 ah, 


where ¥, designates the complex conjugate of y,. 
It is readily verified that the scalar product has the following 
properties: 
1) (X,X) > Of X F 0; (X, X) = OW X = O. 
2) (X,Y) = (¥, X). 
3) (X, +X, Y) = (M1, ¥)+(X2, ¥). 
4) (aX, Y) = a(X, Y). 
5) (X, Yat Yo) = (X, Yi) +(%, Ya). 
6) (X,aY) = a(X, Y). 
In addition, V(X, X) is called the length of the vector. In what 


follows we shall designate it by ||X]|. 
Besides the n-dimensional complex space introduced above, it is 
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useful to consider also an n-dimensional real space, i.e., the aggregate 
of vectors with real components. 

In real space the scalar product is equal to the sum of the products 
of corresponding components of the vectors; the length of a vector 
equals the square root of the sum of the squares of its components. 

We shall most often have to deal with real n-dimensional space, 
turning to complex space only as occasion requires. 

1. Linear dependence. Avector Y=e,X,+eoXat+ +++ +6,,X,i8 said 


to be a linear combination of the vectors X,, Xo, ... X,,. 
It is easily seen that if vectors Y,,..., Y, are linear combinations 
of the vectors X,,..., X,,, any linear combination y, Y, + +++ +y,¥; 


will also be a linear combination of the vectors X),..., Xj, 
Vectors X,, Xy,..., X,, are called linearly dependent if constants 
C1, Ca, -. +5 C, exist, not all zero, such that the equation 


(5) 4X +egXgt --- +0,X, = O 


holds. If, however, this equatior. holds only when all the constants 
¢; are equal to zero, the vectors X,, Xo,..., X, are said to be 
linearly independent. 

If the vectors X,,..., X,, are linearly dependent, then at least 
one of them will be a linear combination of the rest. For if, for 
example, ¢,,#0, we find from (5): 


a re 
6) xX, = -2%, at Xy1. 


THEOREM 1. Jf the vectors Y,,..., Y, are linear combinations of 
the vectors X\,..., Xp, and k>m, the former set is linearly dependent, 

The proof will be carried through by the method of mathematical 
induction. For m=1, the theorem is obvious. Let the theorem be 
true on the assumption that the number of combined vectors be 
m—1. Under the condition of the theorem, then, 


Yy = ey Xt +++ tink 


Y, = ey X1 t+ suena + gm Xm 


Two cases are conceivable. 
1. All the coefficients ¢,;,..., ¢ are equal to zero. Then 
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Y,,..., Y, are in fact linear combinations of only the m—1 vectors 
Xo,...,X, On the strength of the induction hypothesis, 
Y,,..., Y, will be linearly dependent. 

2. At least one coefficient of X, will be different from zero. 
Without violating the generality we may consider that ¢,,;40. 

Let us now consider the system of vectors 


¢. 
ng fle A 
ey 


¥i = Y,—2#! ¥,. 


The vectors thus constructed are obviously linear combinations 
of the vectors Xo, ..., X,,,and the number of them is k—1>m-—-1. 
On the strength of the induction hypothesis they are linearly 
dependent, i.e., constants yo,..., y, that are not simultancously 
zero can be found such that 


yo¥ot +++ +yYi = 0. 


Replacing Y3,..., ¥, by their expressions in terms of 
Y,,..., ¥,, we obtain 


yiYitypeYet iad +y,¥5 = 0, 


é ¢ 
where y, = SF Ye-—+++ —— yy, The numbers 7;,..., yy are 
re) 


not simultaneously equal to zero and accordingly Y;,..., Y, are 
linearly dependent. This proves Theorem 1. 

A system of linearly independent vectors is said to constitute a 
basis for a space if any vector of the space is a linear combination of 
the vectors of the system. 

An example of a basis is the set of vectors 


e, = (1,0,...,0) 


(7) 


rr er) 
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for it is obvious that for any vector X= (x, x2,..., x,) we have 
X = xyeytxntat +++ +XC,- 


This we shall call the initial basis of the space. Such a basis is 
not the only one possible—quite the contrary: in the choice of a basis 
one may he arbitrary within wide limits. Despite this, the number 
of vectors forming a basis does not depend on its sclection. In proof 
of this, let Y,,..., ¥, and Z;,..., Z,, be two bases, and assume 
further that k>m. The vectors Yj, . ..-» Y, are linear combinations 
of the vectors Z;,..., Z,- In the light of Theorem 1, Y,,...,¥; 
are linearly dependent, which contradicts the definition of basis. So 
k=m. Furthermore, since the initial basis is constituted by a 
vectors, any other basis will also consist of » vectors. The number 
of vectors forming a basis thus coincides with the dimension of the 
space. 

Let U,,..., U, form the basis of a space. Any vector X will 


then be a linear combination of U,,... U,: 


(8) X = §0,+&,U.+ --- +6,U,,. 
The coefficients of this resolution uniquely define the vector X, for 
if 
X= §,U,+--- +6,U, = &U,+ +--+ +&U,, 
then (€,—&))U,+ +--+ +(&,—&,)U, =0, and accordingly 
&—§&, =0,...,8,-& =0, 


in view of the linear independence of the vectors U,,..., U,,. 

The coefficients £,,..., &, are called the coordinates of the vector 
X with respect to the basis U,,..., U,. Note that the components 
of a vector x,,...,%, are the coordinates of the vector XY with 
respect to the initial basis. 

2. Orthogonal systems of vectors. The non-zero vectors of a space 
are said to be orthogonal if their scalar product equals zero. A 
system of vectors is said to be orthogonal if any two vectors of the 
system are orthogonal to one another. In speaking of an orthogonal 
system, we shall henceforth assume that all the vectors of this system 
are different from zero. 
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THEOREM 2. The vectors forming an orthogonal system are linearly 
independent. 
Proof. Let X,..., X, be an orthogonal system, and let 


6X +eeXot+ ee: +6,X, = 0. 
In view of the properties of the scalar product we have: 


O = (Xy+ +++ +0,X;, X;) 
= 64(Xy, Xj) + 0s He (Xy Xi) +2 bX, Xi) = eX, 


and, since ||X,||?>0,¢;=0 for any i=1,2,...,2. Thus the sole 
possible values for ¢),¢:,...,¢, in the equation ¢,X,+¢.X. 
+ +--+ +¢,X,=0 are ¢;=¢cg=--- =c,=0, ie., the vectors X), 
Xo,...,X, are linearly independent. It thence follows, first, that 
the number of vectors forming an orthogonal system does not exceed 
n, and, second, that any orthogonal system of n vectors forms a basis 
of the space. Such a basis is called orthogonal. If we have, in addi- 
tion, ||X;\|=1, the basis is said to be orthonormal. An example of an 
orthonormal basis is the initial basis. 

From any system of linearly independent vectors X,,..., X,, it 
is possible to go over to an orthogonal system of vectors Xj, ...,X; 
by means of the process spoken of as orthogonalization. The following 
theorem describes this process. 


THEOREM 3. Let X,,...,X, be linearly independent, An 
orthogonal system of vectors X4,..., X, may be constructed that is con- 
nected with the original set by the relations: 


X41 =X 
Xo = XetanX, 


Ce 


(9) 
XE = XytayyAyt os Hoy Xe. 


The proof will be by induction. 
Let Xj, ..., X%_ be already constructed and different from zero. 
We seek Xj}, in the form 


(9’) Xn = XntmiXi+ see + Ym—1Xm-1- 
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Choose the coefficients 71,..-, Ym-1 SO that (X;,, X))=0 for 
j=l,...,m—1. This is easily done, for 


(Xi Xj) = (Xa XF) + (Xj X})- 


I 


Now (Xj, Xj) #0, since X;#0 by the induction hypothesis, and it is 
accordingly sufficient to take 


y= kn X)), 
i= (FX) 
Replacing now Xj, ..., X;,_1 in equation (9') by their expressions 
in terms of X,,..., X,-1, we obtain finally 


Xn = Xp OmiX1 + cos +e, 


m,m— 


1X, -l- 


It remains to be proved that X1,40. But this is obvious, for 
otherwise the vector X,, would be a linear combination of the 
vectors X},...,X,-1, which contradicts the condition of the 
theorem. The basis of the induction exists, since for m=1 
the theorem is trivial. 

One may pass from any orthogonal system of vectors to the cor- 
responding orthonormal system by dividing each vector by its 
length. 

The process described permits of great latitude in the choice of an 
orthonormal basis, for one may pass from any basis to an ortho- 
normal one by orthogonalization and normalization. 

The scalar product of two vectors is very simply expressible in 
terms of the coordinates of these vectors with respect to any ortho- 
normal basis, for, if U,,..., U, is an orthonormal basis, and 


X= €,U;+--- +&,U,, Y=7,U,+ eos +7,U 
then 


(x, Y) 


(G,U0,+--- +6,0,5 mU,+ Se +7,U,) 
3 3, Gln a) = 3 3 bin U) = 3 baie 


Thus the expression of the scalar product in terms of the co- 
ordinates of the vectors with respect to any orthonormal basis 
coincides with its expression in terms of the components of the 


vectors, i.c., in terms of the coordinates with respect to the initial 
basis. 
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3. Transformation of coordinates. Let us elucidate the change in the 
coordinates of a vector that accompanies a change of basis. 
Let ¢), ¢, ..., én and ¢4, é2, ... ¢, be two bases, and let 


ey = ye tagylet +++ +4y16, 


€o = Ayyxl) tazolot +++ +4, 06, 


(10) 


ly = 4 pl1 HAgnlet ++ Fane: 


We conncct with the transformation of coordinates a matrix A, 
the columns of which consist of the coordinates of the vectors 
€1, €,.... €, With respect to the basis e), é9,... é,, i.€., the matrix 


41 Ag eee Gay 


(11) A=|" : 


The matrix A is non-singular, for it has an inverse, by means of 
which the vectors ¢), é2, ..., €, are expressible in terms of the vectors 
Ci nr 

Now designate by x,,..., x, the coordinates of a vector X with 
respect to the basis ¢,¢,..., ¢,, and by xj, xj,...,% its co- 
ordinates with respect to the basis ¢, ¢),..-,¢n- Let us determine 
the relation of dependence between the old and the new coordinates. 
We have: 

X = xyeytxgtat +++ #xt, = Kept xeegt + + tenes 
= 44 (441¢) +aa1e2+ +++ +4y1¢q) 
+5 (4101 +ag0¢e+ +++ +4,2¢,) 
tXn(41nt1 +Gageot +++ +4anen) 
= (Ay x, +Arortet +++ + aiyrn)er 
+ (4o1%4 + ago%g+ +++ +doq%n)ee 
Pieter oar a ce, me ie te? ta Et dow | 
+ (4,1%1 + 4,9%3 + Sey +4,,%n) en» 
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whence, on the strength of the linear independence of the vectors 
ly Cay oo oy Oye 


Kp = AyyXyHayoxgt +++ Hayy Xn 


Xo = Aa,X +dootat +++ +49,x) 
(12) 2 21%) + 420% Qn*n 


x, = 441% + @yoXe + Sp © +a nXne 


The last equations may be written in matrix form. Of course 
the set of a vector’s coordinates can be considered either as a column 
or asarow. In view of the definition of matrix multiplication, we 
can postmultiply a square matrix only by a column, not by a row. 
In future we shall, therefore (except by special stipulation), take 
the coordinates of a vector as a column. Often in arguments where 
the basis is fixed (for instance, when the vector is given in terms of 
its coordinates with respect to the initial basis), we shall ideritify the 
vector with the column of its coordinates. 

Equation (12) may be written in the form 


(13) x = Ax’, 

where 
*1 x1 
Xo x9 

(14) x=]. and x’ =] | 
Xq Xn 


are the coordinate columns of the vector X with respect to the bases 
€1,.--,¢@, and ej,..., é respectively. 

4. Subspaces. A set of vectors ¥CR, such that any linear com- 
bination of the vectors of this set is itself a vector of the same set, is 
said to be a subspace of the space R,. Ifa group of vectors U,,..., Up, 
that are linearly independent—or even linearly dependent—be 
given, then the set of all possible linear combinations of them will 
obviously constitute a subspace. A subspace constructible in this 
manner is said to be the subspace spanned by the system of vectors 
U,,..., U,. 


We shall show that a basis exists in every subspace, i.e., that 
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there is a set of linearly independent vectors, by lincar combinations 
of which one may exhaust the entire subspace. Let us construct 
the basis in ‘the following manner. ‘Take first an arbitrary vector 
X), different from zero, and consider all its linear combinations, i.c., 
all vectors of the form ¢X,. If these exhaust the entire subspace, 
X, then forms its basis. If the contrary is true, a vector X. will be 
found, linearly independent of X,. Consider the set of lincar 
combinations of X, and X49: if they do not exhaust the subspace, a 
vector will be found linearly independent of them, and so forth. 
The process cannot go on interminably, for in the space &,, there 
cannot be more than z linearly independent vectors. We shall thus 
have constructed a finite system of vectors X,,..., X, such that 
their linear combinations exhaust our subspace, i.e., we shall have 
constructed our basis. 

It will be remarked that the reasoning set forth indicates much 
latitude in choice of a basis. However the number of vectors forming 
a basis will not depend on the manner of its selection, in the light of 
Theorem 1. That number is called the dimension of the subspace. 

Note that the set composed solely of the null vector, as also the 
set composed of the entire space, will each be subspaces in the sense 
of our definition. We shall regard them as trivial subspaces. 

5. The connection between the dimension of a subspace and the rank of a 
matrix. We introduce the important concept of rank, appropriate 
to any rectangular matrix A, 


4p Ayu eee Gay 


Ga, Aga «ee Gog 


A= : 


Amt Ang ee > Aang 


Any determinant whose rows and columns “fit” the rows and 
columns of a matrix is called a minor of this matrix. More exactly, 
a minor of order k of the matrix A is a determinant of the Ath order 
formed from the elements situated at the intersections of any 4 rows 
and any & columns of the matrix A, in their natural arrangement. 

The order of whatever non-vanishing minor is of largest order is 
called the rank of the matrix A. In other words, the rank of a 
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matrix is a number r such that among the minors of the matrix 
there cxists a non-zero minor of order r, but all minors of order 
r+1 and higher are equal to zero or are not composible (as in the 
case, for instance, of a rectangular m xr matrix, m>r). 

The following important theorem is valid: 


THEOREM. The maximum number of linearly independent rows of a 
matrix, as also the maximum number of linearly independent columns, coin- 
cides with the rank of the matrix. 

From this theorem it follows directly that the dimension of the 
subspace spanned by the vectors U,,..., U,, equals the rank of the 
matrix composed of the components of these vectors. 

Indeed, if the rank of a matrix whose columns are the components 
of the vectors U,,..., U,, equals r, then of these m vectors r will be 
linearly independent, and these will correspond to the linearly 
independent columns of the matrix; all the rest of the columns will 
be linear combinations of them. Any vector subspace is a linear 
combination of the vectors U;, ..., U,,, which are themselves linear 
combinations of but r selected linearly independent vectors. Con- 
sequently any vector is a linear combination of r vectors, and there- 
fore the rank r of the matrix in question coincides with the dimension 
of the subspace. 


§3. LINEAR TRANSFORMATIONS 


1. Let us associate with each vector X of a space a certain 
vector Y of the same space. Such an association we shall call a 
transformation of the space. We shall designate the result of the 
application of transformation A to the vector X by AX. 

We shall call the transformation A linear if 


1. A(aX) = aAX, for any complex number a; 
2. A(X, +X) = AX, + AXs,. 


We shall define, furthermore, operations upon linear transforma- 
tions. The product of the linear transformations A and B, AB=C, 


34 Basie Material from Linear Algebra 


will be a transformation constituted by the transformations B and A 
in turn, B being completed first, and then A. 

The product of linear transformations is a linear transformation, 
as is readily seen, since 


C(X,+Xe) = A(B(X,+X2)) = A(BX, + BX.) 
(1) = ABX,+ABX, = CX,+CXo, 
CaX = ABaX = AcBX = aABX = aCX. 


The sum of the linear transformations A and B will be a trans- 
formation C which associates the vector X with the vector AX + BX. 
This sum of linear transformations is obviously itself a linear trans- 
formation. 

2. Representation of a linear transformation by a matrix. Let us 
choose, in the space R,, some basis ¢), ¢2,...,¢,- A linear trans- 
formation relates to the vectors of the basis the vectors Ae,, 
Aes, oes Ae,. 

Let Ae,,..., Ae, be given in terms of their coordinates with 
respect to the basis ¢), ¢,.. . €,, i.e., let 


Ae, = 4110, + Ag lo + eee + 4,1¢, 
Aeg = G00) +doeegt +++ +4yet 


(2) 


Cy 


Consider the matrix A, its columns composed of the coordinates 
of the vectors Ae,, Aes, ..., Ae,: 


411 42 + 441, 
Qo oq «w+ Ga 
(3) Ais 23 22 2a 


oe © © © ee 
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We shall show that the matrix A uniquely defines the linear 
transformation. 

Indeed, if the matrix A is known for the linear transformation, 
ie., if Ae}, Aég, ... Ae, are determined, this is sufficient to find the 
transformation of any vector, for if 


X= xyepte +X ens 
then 
AX = x,Ae,+ oes +x,Ae,. 


Hence the coordinates of the transformed vector are easily found, 
for we have 


n n a n 
Y= AX= Fy = > x4 = > Dd arity 
kal 1 imi £51 


whence 
n 
w= 2, Ay Xj, 
i= 
or, in matrix notation, 
(4) y = Ax, 


where y and x are columns of the coordinates of vectors Y and X. 
Conversely, an arbitrary matrix 4 may be connected with a cer- 
tain linear transformation. Indeed, the transformation given by 
the formula 
y = Ax, 


where y and x are, as above, the columns of coordinates of the 
vectors Y and X, is linear for any matrix A. 

The established one-to-one correspondence between transforma- 
tions and matrices is preserved when operations are performed upon 
transformations, for the matrix of the sum of transformations equals the 
sum of the matrices of the summand transformations, and the matrix of a 
product of transformations equals the product of the matrices corresponding 
to the factor transformations. 

3. The connection between the matrices of a linear transformation with 
respect to different bases. We will now elucidate how the matrix of a 
linear transformation changes with a change of the basis of the space. 


1 Note, however, that the matrix of the coefficients in the relations (2) forms a 
matrix which is the transpose of that that we connect with the linear transformation. 
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Assume that from the basis ¢,, ... , ¢, we have passed to the basis 
€i,.-+, e, and let 
ey = Cyne tegegt +++ Hee, 


ey = Cyt  teyvtet +++ +eqoe, 


O 


ln = C11 + Capeo+ wee + lan ene 


The coordinates of any vector of the space will have been changed 
accordingly by the formula 


x= Cx’, 
where 
Cy) Cla oe l14 x) x) 
€ay Cag «2. Ce Xo Xo 
C= cy , rs : x= 2 
. 
Gc tie ha. Gi Xe Xn 


The matrix of the transition, C, is evidently non-singular. It will 
coincide with the matrix of the linear transformation sending the 
basis ¢}, ¢2,..., @, into the basis ¢}, eb, . . . 

Let us now consider a linear transformation A, and let the matrix 
A correspond to it with respect to the basis ¢,, ¢2, ..., ¢,, and the 
matrix B with respect to the basis ¢), en)... 5 &n 

If x is the column of the coordinates of the vector X with respect 
to the basis e,,.. , ¢,, and x’ that with respect to the basis e}, . . . , eh 
y and y’ being the analogous columns for vector Y, we have 


y = Ax 
y’ = Bx’. 
But x=Cx'’, y=Cy’, and therefore 
Cy’ = ACx' 
or 


y’ = Bx’ = C“1ACx’, 


Thus similar matrices correspond to the same linear transforma- 
tion with respect to different bases. Furthermore, the matrix by 
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means of which the similarity transformation is effected coincides 
with the matrix of transformation of coordinates. 

4. The transfer rule for a matrix in a scalar product. Let X and Y be 
two vectors given by their components with respect to the initial 
basis: X=(x),..-5,%,), Y=(y1,-++54,), and let A be a linear 
transformation with matrix A=(a;,). Designate by A* the linear 
transformation with matrix A*, the elements of which are the 
complex-conjugates of their counterparts in A, and which are placed 
in transposed positions: A* =(a;,)* =(a,,') =(@,;). We shall prove 
the following formula: 


(AX, Y) = (X, A*Y). 


In demonstration, we have 





(AX, Y) = 2 p> 4%,9;, = 2 *, 2, 4,4, = (X, A*Y). 


5. The rank of a linear transformation. Let A be a certain linear 
transformation. The set of vectors AX will obviously constitute a 
subspace, which we shall denote by AR,. 

The dimension of this subspace is said to be the rank of the érans- 
Sormation A. 

We shall show the rank of a transformation to be equal to the 
rank of the matrix corresponding to this transformation on any 
basis whatever, ¢1,¢2,...,¢,- Obviously the subspace AR, is 
spanned by the vectors Ae, Aes,..., Ae,. The dimension of AR, 
is accordingly equal] to the rank of a matrix whose columns are com- 
posed of the coordinates of the vectors Ae), Aés,..., Aé,, ie., to 
the rank of a matrix corresponding to the transformation, 

Since the dimension of a subspace does not depend upon the 
selection of the basis, it follows from the foregoing that the ranks of 
similar matrices are equal. 

6. The proper vectors of a linear transformation. By a proper vector 
(characteristic vector, latent vector or eigenvector) of a linear transformation 
A is meant any non-zero vector X such that 


(6) AX = 2X, 


where 2 is any complex number. 
The number A is called a proper number (characteristic number, latent 
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root, or eigenvalue) of the transformation. The spectrum of the trans- 
formation is the aggregate of its proper numbers. 

The proper numbers and proper vectors of the transformation 
may be determined in the following manner. Let the transforma- 
tion A be connected with the matrix A= (a;,) with respect to some 
basis; let the coordinates of the proper vector X, with respect to this 
basis, be x1, ..., %,- 

The coordinates of the vector AX will then be: 


fn n 
[3 aute ++ Sats | 
k=! k=l 


and thus for the determination of x), x2,...,%, and the proper 
number A we will have the system of equations: 


Ax 


GgiX1+dgoXo+ +++ +49,%, = Axe 


411%) +A, aX +++ +41,%, 


(7) . 
4,)X1 + 4yoXy+ °° FO_.X, = he, 
or 
(41, —A)xy + ara%e+ +++ HO4X, = 
(7') 91X1 + (dgg—A)xo+ +++ +4o,x, = 0 


o 8 © © 8 


41%, $F d,9%0+ + °° + (dyn — A) %q = 0. 


This system of homogeneous equations in x,,..., x, will have a 
non-zero solution only in case 


ayy—A ayy 4}, 

491 ago—A a9, =0 
= 3 

Qni a,2 eee Gan — A 


ie., if A is a zero of the characteristic polynomial of the matrix. 
Thus the following is valid: 
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THEOREM. The proper numbers of a transformation coincide with 
the zeros of the characteristic polynomial of a matrix that is connected with 
this transformation with respect to an arbitrary basis. 

From the theorem known as the fundamental theorem of higher 
algebra, we know that every polynomial has at least one zero. A 
linear transformation will consequently have at least one proper 
number, which may be complex even though the matrix of the trans- 
formation be real. In view of the theory of linear homogeneous 
systems of equations, there will be a non-zero solution of system (7) 
for each proper number, i.e., with each proper number at least one 
proper vector is associated. 

Obviously if X is a proper vector of the transformation A, then, 
for all ¢ 40, cX will also be a proper vector of transformation A cor- 
responding to the same proper number. Furthermore if,several 
proper vectors correspond to some one proper number, then any 
linear combination of them will be a proper vector of the trans- 
formation associated with the same number. The set of proper 
vectors corresponding to the same proper number forms a lincar 
subspace. We shall establish that its dimension, /, does not exceed 
the multiplicity of the proper number. Indced, let X,,..., X; be 
linearly independent proper vectors corresponding to the same 
proper number 4,. Construct a basis of the space Xj,..., Xp 
having taken as the first / vectors the vectors X;,..., X;. With 
respect to this basis the linear transformation under consideration is 
connected with a matrix whose first { columns have the form 


a4, 0 ... 0 
0 a4... 0 
(7b) ee 
00 ... A 
0 0... 0, 


for AX,;=1,X,,..., AX,;=4,X;. Now (A—A,)! is a factor of the 
characteristic polynomial of this matrix, and accordingly 4, is of 
multiplicity & not less than J, i.e., 2<k. It would naturally be sup- 
posed that /=4, i.c., that to a k-multiple root of the characteristic 
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polynomial there correspond k linearly independent proper vectors. 
But this is in fact not true. In reality, the number of lincarly inde- 
pendent vectors may be less than the multiplicity of the proper 
number. 

Let us confirm the preceding statement with an example. Con- 
sider the linear transformation with the matrix 


3 1 
A= : 
0 3 ] 
Then |A—Al| =(A—3)2, and thus 4=3 is a double root of the char- 
acteristic polynomial. 


The system of equations for determining the coordinates of the 
proper vector of the transformation A will be: 


3x, +%2 = 3x, 
3x = 3x9 


whence x, =0, and thus all the proper vectors of the transformation 
in question will be (x,, 0)=+x,(1, 0). So in this instance only one 
linearly independent vector is associated with a double root. 

Generally speaking, the coordinates of a proper vector on the 
chosen basis are to be determined from the system (7) of linear 
equations, in which for 4 the proper number 4, is substituted. But 
as is known from the theory of systems of linear equations, the num- 
ber of linearly independent solutions of a homogeneous system equals 
n—r, where r¢ is the rank of the matrix composed of the coefficients 
of the system. Therefore if r denotes the rank of the matrix 4—AJ, 
f=n—r. Thus n—rgk, and the equality does not always hold.! 

In case the basis does not change in the course of the argument, 
we shall often identify the linear transformation with the matrix of 
the linear transformation with respect to this basis, and any vector 
of the space with the column of its coordinates. 

On this agreement, it makes sense to speak of a proper vector of a 
matrix, understanding by this a column « satisfying the condition 


Ax = he. 


We remark that if a proper number of a real matrix is complex, 
IV. 1. Smirnov, [1]; A. G. Kurosh, [1). 
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the coordinates of an associated proper vector will be complex. A 
vector whose coordinates are the complex conjugates of those of a 
given proper vector of a real matrix is also a proper vector of that 
matrix, and is associated with the complex conjugate proper num- 
ber. To convince oneself of this, it is enough to change all numbers 
in the equation Ax =x into their complex conjugates. 

7. Properties of the proper numbers and proper vectors of a matrix. We 
shall establish several properties of the proper numbers and proper 
vectors of a real matrix. 

First of all we note that a matrix and its transpose have identical 
characteristic polynomials and consequently identical spectra. 
This is evident since |A’—AJ|=|A—AI|, on the strength of the fact 
that a determinant is not altered when its rows and columns are 
interchanged. 

Now let 4, and A, denote distinct proper numbers of the real 
matrix A, and 4, the complex conjugate of 4,. As we saw above, 
i, is also a proper number of matrix A, and thus also of the trans- 
posed matrix A’. Let X, be the proper vector of the matrix A 
belonging to the proper number 4,, and X‘ the proper vector of the 
matrix A’ belonging to the proper number /,. We shall show that 
X, and X} are orthogonal. 

With this object in view, let us form the scalar product (AX,, X}) 
and reckon it by two methods. 

By one method we have 


(AX,, X1) = (4,X,, X1) = A(X, X)). 


On the other hand, since matrix A is real, we have A* =A’, and 
therefore 


(AX,, x) = (X,; AX’) = (X,, 1,X') = A(X, X)). 


Thus A,(X,, X}) =4,(X,, X/). But the condition was that 4,#4,, and 
therefore (X,, X/) =0, which is what was required to be proved. 
In case all the proper numbers are distinct, the demonstrated 
property gives n2—n relations of orthogonality between the proper 
vectors of matrices A and A’. We shall later return to these 
properties in more detail. 
For a real symmetric matrix, the properties of orthogonality are 
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considerably simplified, thanks to the fact that all its proper numbers 
are real. In proof, letting A and X be respectively proper number 
and vector, we have (AX, X)=A(X,X); (AX, X)=(X, A’X) 
=(X, AX) =(X, AX) =A(X, X). Thus (A—/)(X, X)=0, and as 
(X, X)>0, A=A, i.e., 2 is real. 

From the reality of the proper numbers of a real symmetric 
matrix it follows that vectors with real components may be taken as 
the proper vectors belonging to those roots; the components will 
indeed be found by solving the linear homogencous system with 
real coefficients. 

The orthogonality property of the proper vectors of a real sym- 
metric matrix is very simply formulated in view of the coincidence 
of the matrix with its transpose and the reality of the proper num- 
bers, viz.: proper vectors belonging to distinct proper numbers are orthogonal. 

8. The proper numbers of a positive-definite quadratic form. A homo- 
geneous polynomial of the second degree in several variables 
X1,...,4%, is called a quadratic form. We shall consider only those 
with real coefficients. Any quadratic form may be written as 


P(x), Kay eeny xn) = Ay XjXg, 


8, 
i, kel 
where 4;, = 4;;. 

A quadratic form is said to be positive-definite if its values are 
positive for any real values of x, . . . , x,, not all zero simultaneously. 

It is evident that the diagonal coefficients of a positive-cefinite 
form are positive, for 


a;, = H(1,0,..., 0), an, = (0, 1,...,0),.. 
a,, = P(0,0,..., 1). 


Denoting by X the vector with components (x,,...,%,), We 
may write a quadratic form as (AX, X), A being the matrix com- 
posed of the coefficients of the form. This matrix is symmetric, on 
the strength of the definition. The proper numbers of the matrix 
are called the proper numbers of the quadratic form. In view of 
the previous results, all proper numbers of a quadratic form are real. 

We shall show that if @ quadratic form is positive-definite, ils proper 
numbers are positive. 
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In demonstration, let X be a real proper vector, belonging to A, a 
proper number of the matrix of the form. Then, since the form is 
positive-definite, (AX, X)>0. On the other hand, (AX, X) 
=A(X, X). Thus 


But both numerator and denominator of this fraction are positive, 
and consequently A>0, which is what was required to be proved. 
Let there now be given any real, non-singular matrix A. Ob- 
viously B=A’A is a symmetric matrix, since B’=(A‘A)’=A'A’ 
=A‘A=B. 
We shall show that a quadratic form with matrix B is positive- 
definite. We have, indeed, 


(BX, X) = (A'AX, X) = (AX, AX)>0 


for any real vector YX. 
We shall establish, lastly, that if A is the matrix of a positive- 
definite quadratic form, (AX, X)>0 even for a complex vector X, 
In proof, let X= Y+i1Z, where Y and Z are vectors with real 
components. Then 


(AX, X) = (AY+iAZ, Y+iZ) 
= (AY, Y)+i(AZ, Y) —i(AY, Z) + (AZ, Z) 
= (AY, Y)+(AZ, Z)>0 


‘because (AZ, Y) =(Z, AY) =(AY, Z). 
In complex space, instead of the quadratic form one deals with an 
Hermitian form, an expression of the type 


TMs 


GX ips 


under the condition that a,;=4,;. 

The matrix of an Hermitian form is called Hermitian (or Hermitian 
symmetric) ; a linear transformation with an Hermitian matrix relative 
to an orthonormal base is called self-conjugate. It is obvious that 


2 4;%;%, = (AX, X). 


44 Basic Material from Linear Algebra 


To show that all the valucs of an Hermitian form are real, we 
have only to note that 


(AX, X) = (X, A*X) = (AX, 2). 


If all the values of an Hermitian form are positive, it is called 
positive-definite. 

It can be shown that the proper numbers of an Hermitian matrix are 
real. The proper numbers of a positive-definite Hermitian form are positive. 

9. The reduction of a matrix to diagonal form. Let us consider the 
matrix A all of whose proper numbers, 4,,..., 4,, are distinct, and 
the transformation A connccted with it with respect to the initial 
basis. It will have » distinct proper vectors X,,..., X,. We shall 
show that the vectors X,,..., X,, are linearly independent. 

Assume the contrary: let the vectors X),...,X, be linearly 
dependent. Without detriment to the generality we may assume 
that the vectors X,,..., X,, where k<n, are linearly independent, 
and thus that the vectors X;4;,..., X, are linear combinations of 
them. In particular, let 


k 
(8) X= > 


then 


On the other hand, 


k 
AX, = 4,X, = 2 AeiXiv 
int 
whence 


(9) 5 (ajax = 0: 


But 4,#A;, on assumption. Thus, since the vectors X; are linearly 
independent, all the coefficients c, equal zero, the therefore X, =0, 
which contradicts the definition of a proper vector. So the vectors 
X,, Xo,..., X, are linearly independent. Let us adopt them as a 
new basis of the space. With respect to the new basis the linear 
transformation A will be connected with a matrix whose columns 
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are composed of the coordinates of the vectors AX,, AX, ..., AX, 
with respect to the basis X), Xo, ..., X,- 
But 
AX, = AX 


and the matrix of the transformation on the new basis will conse- 
quently be diagonal: ' A, do, ... A, + 

So the linear transformation A has, with respect to the initial 
basis, the matrix A, and with respect to the basis of the proper 
vectors, the diagonal matrix 4), a2,... 4, . Accordingly, on the 
strength of what has been noted above, 


(10) VAAV =", dn... A 


os 


ny 
where V is the matrix whose columns are the coordinates (with 
respect to the initial basis) of the proper vectors. 

Observation, If the proper numbers of a matrix are of multi- 
plicity greater than one, but to each proper number there correspond 
as many proper vectors as it has multiplicity, the matrix may also be 
reduced to diagonal form. This will be the case, for example, with 
symmetric matrices: it can be proved that to each proper number of a 
symmetric matrix there correspond as many linearly independent proper vectors 
as the multiplicity of the proper number. Moreover, the linearly inde- 
pendent proper vectors belonging to a single proper number may be 
subjected to the orthogonalizing process. We have seen, too, that 
the proper vectors of a symmetric matrix that belong to distinct 
proper numbers are mutually orthogonal. Thus for a symmetric 
matrix it is possible to construct an orthogonal system of proper 
vectors forming a basis for the whole space. 

The question of the transformation of a symmetric matrix to 
diagonal form is closely connected with the theory of quadratic 
forms. 

10. The proper numbers and proper vectors of similar matrices. It has 
been established that similar matrices have identical characteristic 
polynomials, and consequently identical spectra of proper numbers. 

We have explained the geometrical cause of this circumstance, 
viz.: similar matrices may be regarded as matrices of one and the 
same transformation, referred to different bases. Therefore the 
proper vectors of similar matrices are the columns of the coordinates 
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of the proper vectors of the transformation under consideration, 
with respect to different bases, and are thus connected by the 
relation x’=C-!x, C being the matrix of transformation of co- 
ordinates. This circumstance may be verified formally: if Ax = Ax, 
(C-1AC) (C-1x) = 1(C-1x). 

11. The proper numbers of a polynomial in a matrix. Let A be a 
matrix with proper numbers 4,,...,4,, and let (x) =a9+4a),x 
+ +++ +4," be the given polynomial. Then the proper numbers 
of the matrix g(A) will be p(A,), p(A2),---, 9(A,). 

This is readily established for a matrix all of whose proper numbers 
are distinct. Indeed, such a matrix can be reduced to diagonal 
form by a similarity transformation: 

A= c-"), As, aey 4, C. 
Accordingly, 
g(A) = C-19gl Ay, do, 2 ay C. 


’ ts ) bi 
But 


(Ay, yoy An) = oar), P(Ag)s- +5 P(An)_p 
which follows from the fact that 


we ren ="H,..., aby 
Consequently 


pA, ceey A, )) = Pare ees uy 


rr | ey 
> %Ay,..-, > adn. 
kad k=0 | 


Tp(Ar), p(s), «+ P(A) ). 


Thus the matrix g(A) is similar to the matrix "(A)), . . . .p(A,) 4 
and accordingly its proper numbers are (A), p(Ae),..-. 9(A,)s 
Q.E.D. ‘ 

This result remains true for any matrix, of which one may readily 
convince oneself, for example, by considerations of continuity. 

We particularly note that the proper numbers of the matrix A‘ 
are As, 

12. The normalization of the proper vectors of a matrix. The second 
group of orthogonality relations. Let A be a real matrix whose proper 


Ml 
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numbers, A), Ao, ..., A,» are distinct, and let X,, Xo, ..., X, be the 
proper vectors corresponding to them. As we saw, the transposed 
matrix, A’, has the same proper numbers. Let Xj, X9,..., X%, be 
the proper vectors of the matrix A’, and their enumeration be so 
chosen that X, and X; belong to complex-conjugate proper numbers. 
We established above that the following orthogonality relation 
holds: (X{, X;) =0 for ij. We shall show now that, having chosen 

1» X9,-.++,Xny in any manner (they are determined but for a 


numerical multiplier), we may normalize the vectors X}, X,...,.X, 
so that (Xj, X;)=1. 
In demonstration of this, the vectors X,, Xo, ..., X, are known to 


be linearly independent, and they accordingly form a basis of the 
space. Resolve X; in terms of this basis: 


Xi = Xi tyoket +++ +yX,- 
Forming the scalar product (X;, Xj), we obtain 
(Xi XD = XE Mt oo $y(Xh MYA te KS KX) 
yi(Xjs Xi), 
whence we conclude that (Xj, X;) =a;#0, for (Xj, X{) > 0. 
Adopting instead of the vectors X,,...,X, the vectors 


] : : soda : 

— X,...,— X,, we arrive at the required normalization, since 
a 

a 


1 ] 
(x; = %i) = a, (Xi, X;) = 1. 

From the relations of orthogonality and normality set forth above, 
we may derive another group of relations between the components 
of the proper vectors of the matrix A and its transpose. 

Form the matrices 


/ Fk tA 
Miy0 Xo, 2 Xn M11 F120 2 My 
CA 
“12 Xoo Xo; Xa X9, 
x’ and X = 7 
, , ta 
Min X2n +++ Ann al *n2 Xan 


The columns of the matrix X are composed of the components of 
the vectors X,,...,X,. The rov's of the matrix X’ are composed 
of the numbers complex-conjugate with the components of the 
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vectors X},...,X,. (We observe that the numbers that are the 
complex conjugates of the components of the vectors X},..., X% 
are the components of the vectors Xi, ...,X,, which will also be 
proper vectors of the matrix A’ belonging to the proper numbers 
Ai,-.-,A,- Thus the ith row of the matrix X’ and the ith column 
of the matrix X are composed of the components of proper vectors 
of matrices A’ and A, belonging to the same proper number A;, and 
not to proper numbers that are complex conjugates of each other.!) 
It is readily seen that: 
(11) XX =], 
for we have the element of the ith row and jth column of the matrix 
X'X equalling > x4i%j = (Xj, X}) =4;;, where 6;; is Kronecker’s 
kal 
delta: 
0, ij; 
bi = 1, i=. 


Thus X’ and X are mutually inverse matrices, and accordingly XX’ 
is likewise equal to J. This gives a second group of orthogonality 
and normality relations between the proper vectors of the matrices 
A and 4’, viz.: 
(12) { samt xiatjet tet bXiXy =O (AS) 
XXjitxperjet ++ +Aiarji_ = I. 

1 [This may perhaps be clarified by a summary in matrix notation. By the 

definitions at the beginning of Paragraph 12, 
AX, = AXp qd) 


AX = AX). (2 
From (2), a 


4X, = 4X; (2a) 
i.e, A'Ri = AR, for Ais real, (2b) 
Now by definition of the matrix .¥’, 
Xi 
oll Re 
R, 


From which, with (2b) and (1) in view, the author’s statement is confirmed.] 
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Thus for a matrix of the second order the ordinary conditions of 
orthogonality may be written in the form (we preserve only one 
index of the components of the proper vectors, designating the first 
components by x, the second by y): 
xx tyiy2 = 0 
oxi tyoy = 0, 
and the normality conditions: 
amtyin = 1 
xo%2+yoye = 1. 
The new relations will be 
xy itxeye2 =O xx) +x—x2 = 1 
yixityers =O yi tyoye = 1. 


Observation. In case the proper numbers of the matrix are 
multiple, but to each proper number there correspond as many 
linearly independent proper vectors as the multiplicity of the root, 
these indicated properties of the proper vectors hold as before. 


§4. THE JORDAN CANONICAL FORM 


We have proved above that if a matrix has n distinct proper 
numbers, it may be brought into diagonal form by a similarity trans- 
formation. Given the presence of multiple roots, however, such a 
transformation may not always be possible. Nonetheless the problem 
of reducing the matrix to as simple a form as possible by a similarity 
transformation can well be posed. The problem is equivalent to 
discovering a basis with respect to which the linear transformation 
connected with the given matrix would have a matrix of simplest 
form, and the latter proves to be the Jordan canonical form. 

Proof of a fundamental theorem to the effect that any matrix may 
be brought into the Jordan canonical form by a similarity trans- 
formation is rather complicated and we will not dwell on it here.! 
We shall limit ourselves to a description of this canonical form. 

1 [See Frazer, R. A., Duncan, W. J., ano Corrar, A. R., [1], § 3.16 for the 


gist of the proof, or, for the proof proper, Turnputi, H. W., AND Artxen, A. C., 
(1], Chapters V-VI.] 
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A matrix of the following form is called a canonical box: 


4,0 0... 0 0 
1 4, 0 0 0 
014 0 0 
0 0 0 1 4, 


On its principal diagonal the single number A; is everywhere to be 
found; directly under the diagonal (in the subdiagonal) are disposed 
elements that are all units; all the rest of the elements are zero. 

A canonical box cannot be simplified by utilizing a similarity 
transformation. It is obvious that a canonical box has the sole 
multiple proper number 2;. It may be easily verified that a 
canonical box has only one proper vector. The minimum poly- 
nomial of a box coincides with its characteristic polynomial, viz., 
it equals (A; — A)" where m; is the order of the box. 

The Jordan (classical) canonical form is a quasi-diagonal matrix 
composed of canonical boxes: 


40 0... 0 


1 a 0 0 
o1l1A 0 
0 0 0 Ay 


-» © 
~ Oo oO 
oo Oo 
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It is admissible that the same number 4; appear in several canonical 
boxes. 

All the numbers 4; figuring in the different boxes are proper 
numbers of the canonical matrix, and the multiplicity of a proper 
number equals the sum of the orders of the boxes in which it figures 
as diagonal element. In proof of this, by the theorem concerning 
the determinant of a quasi-diagonal matrix, the characteristic poly- 
nomial of the canonical matrix equals the product of the character- 
istic polynomials of the separate boxes, each of them equal to 
(A; — A)", where 4; is the proper number and m,; is the order of the 
ith box. Hence follows directly the statement under proof. 

The determination of the Jordan boxes for a given matrix 
A presents certain difficulties. _ The characteristic polynomial 
TI(a;—A)" coincides with the characteristic polynomial of the 
original matrix, and it is consequently possible to find it without 
knowing the canonical matrix itself. Nonetheless, a knowledge of 
the characteristic polynomial still does not make possible the com- 
plete determination of the canonical form, for a proper number 4; 
of multiplicity k; there may correspond several Jordan boxes con- 
taining this number as a diagonal element, and regarding them only 
the sum of their orders will be known, not the order of each box in 
particular. If the canonical form is to be fully determined, a know- 
ledge of the “elementary divisors” of the matrix must be drawn 
upon. 

Designate by D;(A) the greatest common divisor of all the minors of 
the ith order of the determinant |A—AJ|. In particular, D,(A) 
coincides with the characteristic polynomial. It can be proved that 
all D;(A), as D,(A), are common to the class of similar matrices. It 
can be proved, moreover, that D;_,(A) divides D;(A)}. 

Put 


D(a) 
D;_(A) 





= E,(a); 
obviously 


D,(a) = 1 £,(A). 


1 [See, e.g., TURNBULL, H. W., AND ArrKen, A. C., [I], p. 23 ff] 
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D,{A) 


It turns out, moreover, that £,(A) =B 
na~l 


is the minimum poly- 


nomial! of the matrix. 
Resolve £;(A) into linear factors. Then 


E,(a) = I (A;— amu. 
a 


n 
Here s denotes the number of distinct proper numbers, > m,;;=k;; 


iva] 


3 n 
2 2 m,;=n. It is obvious that among the exponents m;; only 
jalizl 


some will be different from zero. 

The binomials (A;—4)™u are known as the elementary divisors of 
the matrix 4. A knowledge of the elementary divisors permits us 
to construct the canonical form, viz.: the Jordan boxes are con- 
structed by starting from the numbers 4;, and the orders of these 
boxes are equal to the exponents m;;. The number of boxes con- 
taining A, equals the number of exponents m;; not equal to zero. 

In case the elementary divisors are linear, i.e., if all the non-zero 
exponents m;; are equal to one, the Jordan boxes degenerate into 
diagonal elements, and the canonical form turns out to be simply a 
diagonal form, wherein, of course, a single proper number will 
appear as often as a diagonal clement as it has multiplicity as a zero 
of the characteristic polynomial. 

The converse is also obvious, for it is clear that if a matrix can be 
brought to diagonal form, its elementary divisors are linear. ‘There- 
fore matrices with distinct proper numbers, as also symmetric matrices, have 
linear elementary divisors. 

Ifall the elementary divisors (4; — 4) are relatively prime (which 
occurs only in case D,_,(4)=1), each proper number appears in 
only one canonical box, and the order of the box equals the multi- 
plicity of the corresponding proper number. Only in this case does 
the minimum polynomial coincide with the characteristic poly- 
nomial. 

Let us now consider the matrix transforming a given matrix A into 
canonical form. With this object in view we introduce into the 
discussion the linear transformation connected with the matrix A 
with respect to the initial basis. Then the columns of the trans- 
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forming matrix will be components of the vectors of that basis in 
terms of which the linear transformation in question is describable 
as a canonical matrix. 

Let this canonical matrix have the form 


A, 
As 
‘5 
where 
4,0 ... 0 
Wie 1 a, ... O 
0 0 ... 4 


and the order of A, equals m,; let US), UM,..., un, ony, 


..-, U be the corresponding basis. Then the following formulas 
for the transformation hold: 


AUY = 2,U24UP 
AUD = 1,U2+UP 


oe 8 e 8 


AU =,U 4YU 


Mey My—-1 ie 
(ry (ry 
AU® = 1,U% 


for allr=1,...,5. 

We see that among the vectors of the canonical basis are the 
proper vectors of the given matrix, one per box. It can be proved 
that with this all linearly independent proper vectors of the matrix 
are exhausted, and consequently the number of linearly inde- 
pendent proper vectors of the given matrix equals the number of 
canonical boxes in its canonical form. In particular, the number 
of linearly independent proper vectors belonging to a given proper 
number equals the number of canonical boxes containing this 
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number, It is not of greater multiplicity than the proper number, 
and is equal to this multiplicity in case, and only in case, all boxes 
containing the given proper number are of order 1, i.e., when the 
corresponding elementary divisors are linear. 


§5. THE CONCEPT OF LIMIT FOR 
VECTORS AND MATRICES 


Let a sequence of vectors X¥, X¥®@,..., X%,... with com- 
ponents (x{,..., x),..., (x@,..., x),... be given. If a 
limit exists for each component: lim x) =x,, the vector -¥, with 

s+) 


components x;,...,%,, is called the limit of the sequence X"), 
X@,..., XX), ... and the sequence itself is said to be convergent to 
the vector X. This is written in the form X)— X or lim X= X, 


kava 
In the same fashion, given a sequence of square matrices A), 


A®,.,.,A#,... with elements (a{)), (ai?)),..., (af), ...,the 
matrix A with elements a;; = lim aff is called the limit of the 
k-o® 


sequence, if all these limits exist. 
In accordance with such a definition of a limit, an infinite series of 
vectors X) 4 X(2)4., 6.4 Xb) 4 «+ is said to be convergent if lim (X™ 


rae 
+X(2)4 ... 4X) exists; this limit is called the sum of the given 
series. Obviously it is necessary and sufficient for the convergence 
of a series of vectors that all the series composed of their correspond- 
ing components, i.e., components bearing the same indices, con- 
verge; the sums of these series are the components of the sum of the 
series of vectors. 

The concept of the convergence of a series of matrices is defined 
analogously. 

In applied questions it is usually important to judge not only the 
convergence of a sequence or series, but also to judge the rate of 
this convergence. With this object in view, the introduction of the 
norms of vectors and matrices is quite useful. A norm may be 
introduced in different ways, and in different cases one or other 
norm will prove to be most convenient. 
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Generally, the norm of a vector X is an associated non-negative 
number |/.X|| satisfying the following requirements: 


1) ||X|| > 0 for X # 0 and {Oj = 0; 
2) {lcX|! = |e| [|X|] for any numerical multiplier c; 
3) WX+ Yi < (XI+I/ YI] (the “triangular inequality’’). 
From requirements 2) and 3) it is readily deduced that 
x= vy > | 1xi-w7i. 
Indeed, we have 
WX] = |]X-Y¥+¥il< X— YI+I¥ Il 


and therefore 
|X—Y|] > |XI-I¥Il. 
But 
[X-Fi] = ||¥—X] > | ¥—-lXI. 
Consequently 


|X —¥l| 2 





WxI-U¥1i| 


We shall henceforth make use of the following three ways of 
assigning a norm: if X= (x), xo, .-. 5 %_q), 


I. |X] = max |x;| 
IN. WX ny = [il + [ae] + ++ +) 
WD. Xn = Vea]? + leek? + + + + |x)? 


It is obvious that for all three norms all the requirements |)—3) 
are fulfilled. 

The concept of the norm of a vector generalizes the concept of the 
length of a vector, since for length all the requirements 1)—3) are 
fulfilled. The third norm introduced by us is indeed none other 
than the length of the vector. 

Furthermore, it is easily established that a necessary and sufficient 
condition that the sequence of vectors X‘*) converge to the vector X 
is that ||X) — X]|->0 for each of the three norms indicated. For the 
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first norm this is obvious. For the second and third norm this 
follows from the obvious inequalities 


WX < Xl < allX lh 
WX < UX < VallX|h- 


It is easily shown that for convergence of a sequence of vectors 
X® to a vector X it is necessary and sufficient that ||X—X||>0, 
whatever norm satisfying conditions 1)—3) we may choose. Here, 


if XW>X, |X WIXI, for | |XX |<|X- X00. 


In an analogous fashion, the norm of a square matrix 4 is a non- 
negative number ||4|| satisfying the conditions 


1) [ld] > Oif A # 0 and || = 0; 
2) fleall = lel IAlls 

3) ||A+ Bil < |All +115); 

4) ||ABll < |All III. 


Just as in the case of the norms of vectors, the condition ||A“) — A[]+0 
is necessary and sufficient in order that A‘)—A, and just as in the 
case of the norms of vectors, it follows from A®—-A that ||A‘:j|->]|Al]. 
The norm of a matrix may be introduced in an infinite variety of 
ways. Because in the majority of problems connected with esti- 
mates both matrices and vectors appear simultaneously in the 
reasoning, it is convenient to introduce the norm of a matrix in such 
a way that it will be rationally connected with the vector norms 
employed in the argument in hand. We shall say that the norm 
of a matrix is compatible with a given norm of vectors if for any 
matrix A and any vector X the following inequality is satisfied: 


AX] < All IAI 


We will now indicate a device making it possible to construct the 
matrix norm so as to render it compatible with a given vector norm, 
to wit: we shall adopt for the norm of the matrix A the maximum 
of the norms of the vectors LX on the assumption that the vector Y 
runs over the set of all vectors whose norm equals unity: 


I!A]] = max ||AX]. 
1X1 
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In consequence of the continuity of a norm, for each matrix A 
this maximum is attainable, i.c., a vector X) can be found such that 
|Xoll = 1 and ||AXol|=|'All. 

We shall prove that a norm constructed in such a manner satisfies 
requirements !)—4), set previously, and the compatibility condition. 

Let us begin with the verification of the first requirement. 

Let A¥0. Then a vector X, [|X||=1, can be found such that 
AX#0, and accordingly ||AX||#0. Therefore |{Al Ba i ae || #0. 


If, however, A= 0, ||Al| = max |' 0X] =0. 
Wiel 


Second requirement. On the strength of the definition, |[cAll 
= max ||cAX|]. Obviously ||¢AX|| = |e] [[AX]] and thus 


el aa lel AX] = |e] max [AX] = |e] All. 


Let us verify, furthermore, the compatibility condition. 
! 


Let Y#0 be any vector; then X =I Y will satisfy the condition 


that ||X|[=1. Consequently 
HAY || = ACIP UX) = WIMAX < WYINAIL. 


Third requirement. For the matrix A+B find a vector Xo such 
that ||4+ Bl|=||(4 +B) Xl] and ||Xo]|=1. Then 


||A + Bl| = |(4+B)Xoll 
= ||AXo+BXoll < ||AXoll+||BXoll < |All | Xoll +11Bll |lXoll 
= 4+ I14I- 


Lastly, the fourth requirement. For the matrix AB find a vector Xo 
such that ||Xo||=1 and ||ABX9||=||ABl|. Then 


|ABl| = ||ABXol| 
= ||A(BXo)Il < HAll IBXoll < |All |BIl | Xoll 
= |All BI. 


We have verified the satisfaction of all four requirements and the 
compatibility condition. A matrix norm constructed in this manner 
we shall speak of as subordinate to the given norm of vectors. It is 
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obvious that for any matrix norm, subordinate to whatsoever 
vector norm, |{/||=1. 

Let us now construct matrix norms subordinate to the three 
norms of vectors introduced above. 


I. Xl; = max |x;,|. 
8 
The matrix norm subordinate to this vector norm is 
n 
|All; = max > laial- 
é k=l 
In proof, let {X|]=1. Then 
n an a 
AX] = max | 2 a;,x,| < max 2 lajsl |x| < max 2 la;,\- 
‘ = ioks i ok 


Consequently 
max ||AX|| < max > Ja;,|- 
i kel 


WXi= 4 


We shall now prove that max ||4X'j is in fact equal to max 2. |a;,l. 
For this we shall seer - vector Xo such that |Xyl=1 and 
||AXol| = max 2 |a;,|. Letting 2. la;,| attain its greatest value for 
i=j, and then taking as the component x{° of the vector Xo: 
x? =H, if a,#0, and x=1 if aj,=0, we have, obviously, 


1Xal=1- Furthermore, 
| 3, | < 3 leul< 3 aul forix 
and 
| aus] = Sout 
Consequently 


an n a 
irae Wo a ~ 2 ajn| = ve 2 ai |. 
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Thus ||AXql|=max 2 a;,|, Q.E.D. 
U1. Xin = 3 lab 
The matrix norm subordinate to this vector norm is 
Alla = ag > laiel- 
In proof thereof, let [‘X||=1, then 


AX] 





a a n n 
>| > aie| < > Dd laced bral 
ier tasr " i=1 eS 


IN 


3, bul [3 lal] 


IN 


a a n 
[max § laul] 3 lal < max 3 lal. 
& i=] ke} & jel 


fa 
Now let us take a vector Xo of the following form: let > fa;,{ 
iet 
attain its greatest value for the column numbered j. Put x =0 
for k#j and x =1. Obviously a vector constructed in this manner 
has its norm equal to unity. Furthermore 








AX, = 5 x0) = : | = 5 «ple 
| oll 2, 2, Tj 2, \a;,| la p> |a;,| 
Thus 
max ||AXo|| = a Ra lal, 
Q.E.D. 
II. Win = p> lx]? = (X, X). 


The matrix norm subordinate to this vector norm is 


4th = WAL, 


where A, is the largest proper number of the matrix A’A. In proof, 
we have 


Al = max |[4X]]; 
WX tes 
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but 
\AX|7,, = (AX, AX) = (X, A’AX). 


The matrix A’A is symmetric. Let 2,;2 4. 2--+ 24, be its proper 
numbers and X), X2,..., X, be the orthonormal system of proper 
vectors belonging to these proper numbers, 

Now take any vector X with its norm equal to unity and resolve 
it in terms of the proper vectors: 


X = 0X, +6gXot -++ +6,X,. 
Then 

(X,X) = &+dt+--- +2 =1. 
Moreover, 


JAX]? = (X, A’AX) 
= (6X t+ ++ He,Xy, A,X + +++ +6,4,X,) 
= Ait ++ +A < A(t --- +c?) = A). 
For the vector X= X,: 


JAX|® = (Xy, A°AXy) = (X), 41X14) = A. 
Thus 


max |AX|| = Vay, 
Q ED WXp=1 


We shall now prove several theorems connected with the concept 
of limit. 


THEOREM 1. Jn order that Av—>0, it is necessary and sufficient 
that all the proper numbers of the matrix A be of modulus less than unity. 

Proof. Assume for simplicity that the matrix A can be brought 
into diagonal form: A=CAC-1, where A=! Aj, do, ..., 4, _) and 


Ay, 42,-+-,4, are the proper numbers of matrix A. Then 
Am=CA*C-}, It is obvious that A»=" 47, A, ..., am 'y In order 


that A=—+0, it is necessary and sufficient that A™ +0, for which it is 
in turn necessary and sufficient that all proper numbers A,, do, ..., A, 
have modulus less than unity. 

In case the matrix A cannot be brought into diagonal form, the 
theorem is proved either with the aid of considerations of con- 
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tinuity or by passing to the Jordan canonical form. We shall not 
dwell on the details of this proof. 

The conditions given in Theorem | are inconvenient for testing, 
inasmuch as they require foreknowledge of the proper numbers of 
the matrix A. We shall therefore establish some simpler sufficient 
conditions rendering lim A™=0. 


m—>o 


THEOREM 2. Jn order that An—>0, it is suffictent that any one of 
the norms of A be less than unity. 

Proof. On the strength of the fourth requirement of a norm, we 
have 


ill] < [jdm lll] < [lde-2|| |All? < +++ < [ladle 


Therefore ||A7||>0 if {|Al[<1, and thus, in view of the foregoing, 
An—>0, 


Combining Theorems | and 2, we arrive at the following result: 


THEOREM 3. No proper number of a matrix exceeds any of its 
norms in modulus. 


A, where e is 





Proof. Let |J-[}=@. Consider a matrix B= ! 
ate 


any positive number. We have 
a 
Bl] = ate <i, 


and accordingly B=—+0 as moo. On the strength of Theorem 1 
its proper numbers have modulus less than unity. But it is obvious 


that the proper numbers of the matrix B equal 





1 
oF A;, where A; are 
the proper numbers of the matrix A. Thus al <I, ie., |Aj|<ate. 
Since ¢ may be taken arbitrarily small, |A;| <a. 

THEOREM 4. Jn order that the series 
(!) T+At-+-+ tiimy --- 


converge, it is necessary and sufficient that ~>0 as m—>oo. In such a case 
the sum of series (1) equals (I—A)7). 
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Proof. The necessity of this condition is obvious. We shall 
show that it is sufficient. 

On the strength of Theorem 1, all proper numbers of the matrix A 
are less than of unit modulus. 

Accordingly 

|7-—A| # 0 

and therefore ({—A)~! exists. 

Consider the identity 


(I+ A+A2+ -++ +A*)(T—A) = I— Abt, 
Postmultiplying it by ({—A)~!, we obtain 
It A+A24 +--+ +At = (1—A)-!— Aft1(I— A)-1, 
whence it follows that, as kK->0o, 
I+A+ +++ +Ab>(I—A)-} 
since Aét1-+0, 


Thus 
T+At-++ Akt... =({—A)-!, 
which is what was required to be proved. 

In the light of Theorem 1, the necessary and sufficient condition 
for the convergence of the series (1) is the inequality |A,|<1 for all 
proper numbers of the matrix A. A sufficient criterion of con- 
vergence, in view of Theorem 2, is the inequality ||A||< 1, whatever 
one of the norms be employed. Given that this condition is satisfied, 
it is easy to give the following estimate of the rate of convergence of 
the series (1): 


THEOREM 5. If |All <1, 
|(7-A)-2- (14+ A+ --- +A8)]] < 
Proof. We have: 


(f—A)-1-—(I+ A+ +++ +44) = Abtl 4 Akt24. 22. 
whence 


I[Alle*# 
1—||4ll 





k+1 
WEA) (At EAD < Age ati = 


and the theorem is proved. 





CHAPTER II 


SYSTEMS OF LINEAR EQUATIONS 


This chapter is devoted to three problems that are closely 
related to each other: the problem of solving a non-homogeneous 
linear algebraic system, the problem of inverting a matrix, and the 
problem spoken of as elimination. 

In theory all of these problems are soluble simply enough. How- 
ever, if a matrix of high order is connected with the problem, the 
actual solution requires a great number of computational operations. 

Numerical methods solving the stated problems are divisible into 
two groups: exact and iferative methods. By exact methods we under- 
stand methods that give the solution of a problem by means of a 
finite number of elementary arithmetic operations. The number 
of computational operations necessary for the solution of the problem 
depends only upon the form of the computational scheme and upon 
the order of the matrix defining the given problem. Inexactitude 
in the solution found occurs as the result of the inevitable rounding- 
off of the figures in the course of the computation. Along with this, 
one may run up against the phenomenon of the disappearance of 
significant figures in the course of the computation, as the result of 
the subtraction of two numbers differing little from each other. This 
loss of significant figures may occasion such an important reduction 
in the accuracy of the result that it is often necessary to alter the 
computational scheme because of it, or re-do the work with a greater 
number of significant figures in the intermediate calculations. 


64 Systems of Linear Equations 


The fundamental method of this group is the method based on 
the idea of elimination. The algorithm of this method, with which 
the name of Gauss is associated, consists—when applied to the solu- 
tion of a non-homogeneous linear system—of a chain of successive 
eliminations by means of which the given system is transformed into 
a system with a triangular matrix whose solution presents no 
difficulty. 

Many different computational schemes have been developed and 
are currently employed for all three of the problems mentioned. 

Those schemes known as compact schemes are outstanding among 
these. The compact schemes are based on the possibility of con- 

a 
> ab, 


= all at one 





ducting the calculations of expressions of the form 


time on the modern calculating machine, dispensing with the 
recording of the results of the intermediate computations of the 
separate summands a,b,; this type of computation is spoken of as 
accumulation. By accumulative computation the rounding error is 
naturally reduced sharply and the volume of intermediate results 
to be recorded is considerably diminished. 

A method founded on the idea of bordering also belongs to the 
exact methods. At the root of this method lies the representation 
of a given matrix in the form of matrices situated one within the 
next, starting with a matrix of the first order. 

Iterative methods afford a means by which a system of linear equa- 
tions may be solved approximately. The solution of a system by 
iterative methods is obtained as the limit of successive approxima- 
tions computed by some uniform process. The convergence of 
these approximations depends essentially on the elements of the 
matrix defining the given problem. The rate of the convergence 
depends also on a happy choice of the initial approximation on 
which the iterative process is founded. 

In particular, for the basic iterative processes, there exist matrices 
for which a process converges only slowly, or even diverges. 

The immense advantage of the iterative schemes consists in the 
simplicity and uniformity of the operations to be effected, and there- 
fore in the possibility of completely mechanizing the process of 
computation. 
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In the continuation of this chapter, we shall assume that all the 
given numbers defining the condition of the problem are real. 


§6. GAUSS’S METHOD 


In this section we shall recall to the reader the well-known 
algorithm for solving a system of equations by a successive elimina- 
tion of the unknowns. As has already been mentioned above, it is 
associated with the name of Gauss. For simplicity of reasoning and 
in order to carry the calculation to the end, we shall consider a 
system of four equations in four unknowns. 

Thus let there be given the system of equations 


411%1 +41 9XQ+419%3 + Ay 4X4 = A156 
1 4g1X1 + AgoXy + AogXg + doqX4 = O25 
() 43 1X1 + AyoXe + 433%3 + dgqX4 = O35 
4g 1Xy + gore + A4gXq + O44X4 = O45 


Let us divide the coefficients of the first of equations (1) by the 
coefficient @,,, which we shall speak of as the leading clement, and 
use the designation 


(2) by = St; fod. 
411 


We will have obtained a new equation 
(3) X1 + by 0x9 +51 3x3 + by yxy = dy5. 


Let us now pass on to an auxiliary system, composed of the three 
equations in three unknowns that are obtained as the result of elimi- 
nating x, from the last three equations of (1) by means of equation 
(3). This elimination is easily done by multiplying equation (3) by 
@2), 43), 44, in turn (i.e., by the leading elements of the second, third 
and fourth rows) and subtracting it from the corresponding equa- 
tions of system (1). The coefficients of the new system obtained as 
the result of the elimination of one variable, we shall designate by 
a;.y: 


(4) 455.1 = @;;—4;1)),; ty 2 2. 
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Proceeding, we shall now divide the coefficients of the first equa- 
tion of the new system by the “leading” element agp. 1. 
We shall have obtained the equation 


(5) X_ + bo3.1Xg + bog.1%q = 56-1) 

where 

(6) boyy = EIS f > 2 
29°] 


Continuing the process analogously, we shall pass to a system of two 
equations in two unknowns, whose coefficients have the form 


(7) G59 = Ay —Ajoyboi4, 1,7 2B 3. 


Dividing the coefficients of the first equation of this system by the 
leading element a33.2 and defining 


(8) baj-g = 2; j > 3, 
433-2 
we shall obtain the equation 


(9) xg + bgq-0%4 = d35.-0. 


Finally, a last step will lead us to a single equation containing a 
single unknown with coefficient 44.3. Upon dividing this equation 
by aq4.3, we obtain 


“4 = b45-3 


Collecting all equations with coefficients 5;;.;1, j> i, we shall have 
obtained a triangular system equivalent to the given onc; its solution 
will be the solution of our system. We note that the process 
described is possible only on condition that none of the leading ele- 
ments by which we have, in the course of the work, divided the 
coefficients of the first equations of the intermediate systems, are 
equal to zero. 

So for solving the given system we first construct an auxiliary 
triangular system, and then solve it. The process of finding the 
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coefficients of the triangular system we shall call the forward course, 
and the process of obtaining its solution, the return course, or back 
solution. 

The scheme that we have described we shall call the single-division 
scheme {see Table I). 

We shall say a few words concerning a control (check) that it is 
expedient to employ when computing by the single-division scheme. 
This control is based on the following circumstance. If in the 
system given we makc the substitution ¥;=x,+ 1, then for determin- 
ing the ¥; we will obtain a system with the former coefficients, and 
with its constant side equal to the sums of the elements of the rows 
of the matrix of the coefficients (plus the constant terms). There- 
fore having formed the sums of the clements of cach row (the con- 
trol sums), we shall perform upon them the very same operations as 
upon the rest of the elements of the row. In the absence of com- 
putational blunders, the numbers found must coincide with the 
analogous sums of the transformed rows. The return course is 
controlled by finding the numbers &; and their coincidence with the 
numbers x; +1. 

We will briefly explain Table I. 

The forward course is carried through as follows. Having copied 
the matrix of the coefficients (including the constant terms and the 
control sums), we divide the first row by the leading element and 
copy the result as the last row of the matrix. Next we compute the 
elements a;;.,; i,j 22, of the first auxiliary matrix: taking any ele- 
ment of the given matrix, we subtract from it the product of the 
leading element of the row to which it belongs by the last clement of 
the column to which it belongs. Continuing, the process is re- 
peated. The computation of the forward course is concluded when 
we arrive at a matrix consisting of one row. 

In the return course, one utilizes the rows containing units, begin- 
ning with the last: in the last of these rows, in the column of constant 
terms, we obtain the value of the last unknown, and in the control 
column, the control value. Continuing, the values of the unknowns 
are obtained sequentially, as the result of subtracting from the 
elements of the next-to-the-last column the products of the cor- 
responding 6-coefficients by the values of the unknowns found pre- 
viously. The unit symbols displayed at the end of the scheme aid in 


Tasie Il. The Single-Division Scheme: Symmetric Case 


1.00 | 042 | 0.66 
1.00 0.44 

0.22 

1.00 

0.66 


0.16280 


— 0.13640 
0.56440 


0.11316 | 0.19767 
0.69785 | | —0.15482 
0.53222 

0.22185 
"0.49787 | 

1 





0.37400 


0.53800 
0.70200 
0.45410 


0.49568 


0.62807 
0.71030 


0.73804 


1.48240 
1.03917 
0.04348 
— 1.25780 


1.45360 
1.20320 
1.29280 
1.76493 
1.03871 
1.00547 
1.48844 
1.23591 
2.48240 
2.03916 
1.04348 
~ 0.25779 
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locating the coefficients that correspond to the given x in the neces- 
sary rows. Thus 


(9b) xo = bo5-4 — bog-1%4—b99-1%3 
= 0.45410 —0.19767 - 1.48240 —0.11316- 1.03917 = 0.04348. 


In conclusion we note that the operation of subtraction may be 
replaced by addition if in copying a row with coefficients 6, one 
reverses their signs. 

The number of multiplications and divisions necessary for finding 
the solution of a system of n equations by the single-division scheme 


is equal to q(nt+3n— 1). 


In case the matrix of the coefficients of the system is symmetric, 
i.€., a;;=a;,, we have, obviously, a;;..=a,;.,. We may therefore 
omit writing the elements below the principal diagonal. The 
single-division scheme, as modified for symmetric matrix, is shown 
in Table IT. 

The leading element of the row, omitted in writing the coefficients, 
(and which we need for the computation of the elements of the 
auxiliary matrices) is easily found as the uppermost element of the 
column containing the diagonal element of the given row. The 
control column contains, as before, the sums of all the elements of 
each row, even including those omitted in tabulation. 

In case several equations with the same matrix must be solved, 
their solutions are naturally sought simultaneously, the constant 
terms being entered in adjacent columns. The control sum is 
formed as the sum of the elements of the rows of the expanded 
matrix. The scheme for solution of several equations is given in 
Table ITI. 

Other computational schemes as well are possible for Gauss’s 
method. We mention a scheme of multiplication and subtraction 
in which the coefficients of the auxiliary systems, which we shall 
designate by 4;;.,, i, >, are obtained as the difference of the 
cross-products 


Ajj = Ag Ajj-1 Agger Appar 


With this scheme the forward course requires no divisions at all; 
they are all moved over into the return course. 


Taste III. Single-Division Scheme: Several Equations 


x2 x 


0.42 0.54 


1.00 0.32 
0.32 1.00 
0.44 0.22 
0.42 0.54 


0.09320 0.70840 
0.16280 — 0.13640 


1 0.11316 


0,69785 
~ 0,15482 


0.82360 0.09320 


Xs 
0.66 
0.44 
0.22 
1.00 
0.66 
0.16280 
- 0.13640 
0.56440 
0.19767 
— 0.15482 
0.53222 
= 0.22185 


0.49787 
| 


0.25 
0.45 
0.65 
0.85 
0.25 
0.51500 
0.68500 
0.41889 


0.47596 


0.61680 
0.68204 


0.72239 
1.45096 


1.00394 
0.01847 
= 1.25752 


0.34500 


0.3 
0.5 
0.7 
0.9 
0.3 


0.37400 


0.53800 
0.70200 
0.45410 
0.49568 
0.62807 
0.71030 


0.73804 
1.48240 


1.03917 
0.04348 
— 1.25780 


0.15 
0.30 
0.45 
0.60 
0.15 


0.23700 2.03560 


0.36900 2.08720 
0.50100 aaiphet 
0.28776 2.47159 


0.34218 1.85685 


(2) 
0.45415 2.07643 
0.49033 2.66082 


(6) 
0.53006 2.48838 

{2) 
1.06466 


(3) 
0.72652 3.76964 


— 0.00499 eee 
— 0.94294 es 2.45838" 





Horizontal sums in superior position. 
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The method of pivotal condensation (dominant elements) is quite 
generally encountered. In this scheme the elimination is con- 
ducted as follows. Select the largest element of the matrix -1, a;;, 
which is called the pivot (dominant element), and compute the 
multiplier 


for all ik. Now multiply the pivot row (i-e., the Ath row) of the 
matrix A by the multipliers m, in turn, and add the result to the 
corresponding ith row. As a result one obtains a new matrix of 
which the jth column consists of zeros. Discarding this column and 
the kth row, we go over to the first auxiliary matrix with a number 
of rows less by one. The subsequent eliminations are conducted 
each time by means of the pivotal element of the transformed matrix. 
The sought-for triangular system is obtained by collecting the pivot 
rows. 

In concluding this section we remark that in computing the 
coefficients of the intermediate systems by the single-division scheme 
we may encounter a disappearance of significant figures. If this 
occurs for a leading element this implies an important loss of 
accuracy. 

In such a case the pivotal condensation scheme may be con- 
veniently used to carry the computations through. 

In such modification, the process is always applicable, provided 
the determinant of the system is different from zero. 


§7. THE EVALUATION OF DETERMINANTS 


Gauss’s method, developed in the preceding section for the 
solution of a linear system, also may be applied to the computation 
of determinants. We shall dwell separately on a description of the 
corresponding scheme, since the computation of determinants is 
frequently met with in applications. Let 


bo 0 ed 0 


A =| S22 +e Fn 


Qny 452 -+- 4 


The Evaluation of Determinants 73 


and Ict a,;#0. Remove the clement aj, from the first row. We 
then have, using the symbols of § 6, 


1 bis eee b,, 


Gx, Ang «ee Aa 
d= ayy . 


4,1 yz +++ 4 


Next take from cach row the first row multiplied by the first clement 
of the particular row in hand. We will obviously obtain 


| bie ee fet b1, Aggy 6 ee Aan) 
0 au 1 ao 
an * ant . 
A=a, = ay . 
0 Ayey Bee Gan ana] eee Annet 


With the determinant of the (n—1)th order that remains, we shall 
now deal just as before, provided ay.., #0. 

Carrying through the process, we shall find that the sought-for 
determinant is equal to the product of the leading elements: 


A = 44,4990) «+ Apyen—1 


If at any step it should turn out that a;;.;-;=0 or a;;.;1 is close 
to zero (which would imply the reduction of the accuracy of the 
computations), the order of the rows and columns of the determinant 
could be rearranged beforehand so that a non-vanishing element 
appears in the upper Ieft corner. 

The best result—in the sense of reliability—will be obtained if at 
each step of the process the maximum element (in point of absolute 
valuc) of the matrix whose determinant is being computed is brought 
to the upper left corner. 

The number of multiplications and divisions necessary for the 
computation of a determinant of the nth order is equal to 


alee (n2+n+3). 





In Table IV is given the scheme for computing the determinant 
of the example. 


TaBLe IV. Computation of a Determinant 


0.42 
1.00 
0.32 
0.44 
0.42 
0.82360 
0.09320 


0.16280 
J 


0.54 
0.32 
1.00 
0.22 
0.54 
0.09320 
0.70840 

—0.13640 
0.11316 
0.69785 

— 0.15482 
| 


0.69785 


0.16280 
— 0.13640 
0.56440 
0.19767 


— 0.15482 


0.53222 
— 0.22185 
0.49787 
0.49787 


0.54303 
0.37740 
0.77815 
0.49787 





PL 
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From an inspection of the process of computing the determinant 
we see that, with the exception of the last multiplication, it coin- 
cides with the forward course of the Gauss process, applied to a 
system with the matrix for which the determinant is computed. 

The familiar rule of Cramer (§ 1) shows that the solution of a 


. : 4d; . 
linear system may be found in the form x; =—2;i=1,..., 2, where 


4 

A denotes the determinant of the coefficients of the system, and 4; 
the determinant of the coefficients in which the ith column has been 
replaced by the constant terms. Thus for the solution of the 
system one must compute n+! determinants of the nth order. 

Comparing the Gauss process for the solution of a system with 
that for computing a determinant, we see that the volume of com- 
putations for the solution of a system only slightly exceeds that for a 
single determinant. The use of Cramer’s formula for the numerical 
solution of a system is thus inefficient. Essentially in the Gauss 
method the computation of all the determinants 4 and 4; is effected 
simultaneously, the division by 4 =@,,@99.) . . . @q-,-1 being accom- 
plished gradually, one factor per step. 


§8. COMPACT ARRANGEMENTS FOR THE 
SOLUTION OF NON-HOMOGENEOUS 
LINEAR SYSTEMS 


In § 6 we saw that the solution of a system of linear equations 
by the single-division scheme reduced to the determination of the 
coefficients a;;., of auxiliary systems (including the constant terms), 
and the coefficients 4;;.;_, of the equations of a final triangular 
system. Only the coefficients 5,;.;_; were necessary to us there for 
obtaining the solution of the given system, and the numbers 4;;., 
played an auxiliary role, being necessary only to determine the num- 
bers 4;;.;-, which we shall indeed henceforth denote simply by 4;;, 
dropping the index denoting the number of the step. 

We shal] show! that the numbers 6,;; can be obtained by a process 
of accumulation permitting one to dispense with the computation 
and recording of all the coefficients a;;.,. 


1 P, Dwyer, [1]. 
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Let us single out the elements of the first column of each auxiliary 
matrix, @;;.;-1, 72j, designating them ¢;;, i>. 

Analysing the process of computing the coefficients of the 
auxiliary matrices, we see that 


ai = G3 5h — Giga 1O gy ad -_ G5 5-4-1 — 62g; 


() = G5 5-4-9 — Care — Oa = 


k 
= 4;;— 65161; — Cabos — 6 + 6 4bgy = 4; — p> €514,- 


Thus any element a;;., is expressible in terms of an accumulation of 
the elements we have singled out, c;;, and the numbers 4;;. 

In particular, for the elements ¢;;, i2j, and 5;;, i<j, themselves, 
the recurrence formulas 


Pr 


ix! sa Ms 
Cig = Ajjj-y = aj3— PRL i2j 
(2) 
i-1 
ea a;;— 2 C44; 
b;; a= we! sis] = pani (2 Dae : t <j 
4ij+j-1 Ci 


hold. The constant terms of the transformed system are obviously 
to be determined in accordance with these same formulas. 

The return course remains the same as in the full-length single- 
division scheme. 

Computation by the compact scheme may be conveniently 
arranged as is shown in Table V. 

Here we effect the computation of the elements ¢ and & succes- 
sively anglewisc, beginning with the computation of the elements ¢: 


C11 bin. Bag bi4 Ist step 
Coy 2 lag bos bog —- 2nd step 
C33 : 39 : €33 b34 3rd step 


41} hse C43 


Tasce V. Compact Scheme for the Single-Division Method 





0.09320 
0.16280 

















0.54 
0.32 
1.00 
0.22 
0.54 
0.11316 


; — 0.15482 


Horizontal sums in superior position. 





0.66 
0.19767 
—0.22185* 





0.3 
0.45410 


0.71030* 
1.48240 


1.48240 
1.03917 
0.04348 
— 1.25780 


2.92 
1.76493 


(3) 
1.48844 
(40) 


2.48239 


(40) 
2.48239 


(7) 
2.03916 


1.04348 
(80 
—0.25779 
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© Obtained by rounding to five decimals before performing division, which renders the procedure consistent with that af Table I. In practice this is nut _ | 


ordinarily bothered with, and is not here in the check column. 


~ 
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Here any element is obtained as the difference between the cor- 
responding element a and the sum of paired products: of the elements 
¢ located in the given row (at the left) and the elements 6 located in 
the given column (above). Of course in computing the elements 4 
it is still necessary to divide by the corresponding element ¢. 

Thus 


€43 = 443—C41b13—C4nb05 


0.22 — 0.66 -0.54 —0.16280-0.11316 = —0.15482; 


— 734 —6a1b14 — Candee 


bse cag 
_ 0.22 0.54 -0.66 —0.09320-0.19767 _ _ 9 oo igs, 


0.69785 


We shall say a few words about the control to be applied when 
computing by the compact scheme. Just as before we form a 
column of check sums and perform the same operations upon it as 
upon the column of constant terms. 

In this connection each number of the transformed column must 
coincide with the sum of the elements of the corresponding rows of 
the matrix B, augmented by adjoining the transformed column of 
constant terms. The matrix B is of course the matrix of the coeffi- 
cients of the system obtained from the given one after completion of 
the forward course by the single-division scheme.} 

In computing by the compact scheme it is convenient to use a 
triangle or two rulers in order to fix the attention on the required 
row and column. 


1 [Note that in Table V the units entered stepwise complete the tabulation of 
matrix B by filling in its diagonal; these of course figure in the row-total of the 
check. The return course, the third panel of Table V, is conducted by utilizing 
the same numbers 4,,,;_, as were utilized in Tables I to IIT, in accordance with 
formulas (9), (5) and (3) of §6, i.c., analogous to formula (9b). The units 
arrayed northeast-southwest in the table serve, as there, the same purpose of 
locating the appropriate 4,;.;-1, these being found successively in successively 
higher rows of panel two here, as distinct from Tables I, II and III, where they 
were found at the bottom row of successively higher panels.] 
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§9. THE CONNECTION OF GAUSS’S METHOD 
WITH THE DECOMPOSITION OF A 
MATRIX INTO FACTORS 


It was shown in § 1 that a system of n linear equations in n 
unknowns can be written in matrix form 


(1) AX = F, 


A being the given matrix, nonsingular, VY and # being columns com- 
posed of the values of the unknowns and of the constant terms, 
respectively, and which we shall speak of as vectors. 

The Gauss method, performed with a fixed order of leading cle- 
ments, consists in replacing the given system by an equivalent 
triangular system by combining the equations linearly; this reduces 
to combining linearly the rows of 44 and F. In the course of its 
application, in using the single-division scheme, we are obliged to 
add to the elements of some rows elements proportional to the 
elements of the preceding rows, i.e., to effect upon matrix 4 cle- 
mentary transformations of Type b’ of Paragraph 10 of § 1. 

The result of several transformations of this form, as was shown 
there, is equivalent to a premultiplication of the matrix A by some 
triangular matrix of the form 


Yiu 0 sew 0 
(2) Yor Poe +e. O -f 
7'nl Yat ose Yan 


As the result of these transformations we arrive at a system with 
a triangular matrix 


be bee. tee bee 


(3) B 


Thus [4=B, which is to say d=/*-! B, and consequently the 
matrix A is factorable into the product of two triangular matrices. 
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The compact scheme realizes this factorization. Indeed, J-) =C, 
where the elements of the matrix C are defined by formulas (2) of 
§ 8, for from these formulas it follows that 


jz! i Zant 
Oy; = eg+ Ps cb = 2 Cub, TBS 
(from the formulas for ¢;;), and 
1) é . = 
a5 = > cub tcibiy = p> Cubiz, P< J 


(from the formulas for 4, ;). 
These last formulas state that 


A = CB. 


Since the diagonal elements of the matrix B are equal to unity, 
such a factorization is unique. 

The compact notation for the schemes of the Gauss method that 
differ from the single-division scheme are likewise related to the 
factorization of a matrix into the product of two triangular ones, 
but with another choice of the diagonal clements. 

We remark that the compact schemes fix the order of the elimina- 
tion, and they are therefore applicable only in case all the deter- 
41 fe 
421 Age 
the theorem of Paragraph 11, § 1. 

We shall show that in case the matrix A is symmetric, 


minants a), »-++,» [4] do not equal zero, as follows from 








(4) by = 


oii 


We have, indeed, A=CB, A’ = B'C’, and, since A= J’, 


Co (A 

4, 0 ... 0 1 2. 2 

11 tn Cis 

CB = BC’ =B'f 0 ca ... 0 0} = 


oe 6 © ee e FF He ee el le le 
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Hence, on the strength of the uniqueness of the factorization of the 
matrix A into the product of the two triangular matrices, it follows 
that 


; 2 fat 
11 1 
Cn 
B= 0 1 . = | 
C22 
00... 1 


Thus the elements of 8 are simply the elements of the matrix C, 
divided. This notwithstanding, the compact scheme requires n? 
transcriptions even for the case of a symmetric A. 


§ 10. THE SQUARE-ROOT METHOD' 


In this section we shall show that in case the matrix of the 
system is symmetric, finding the solution may be rendered still 
easier, since in such case the matrix can be resolved into the product 
of two triangular matrices of which onc is the transpose of the other. 

Thus let 


(1) A=S'S, 


where 


(2) S= a ee os 
0 O ... S, 


Let us determine the elements of the matrix S. In view of the rule 
for the multiplication of matrices, we have 


G35 = $1581; +5ajSap+ - 0° +5;iSij0 E <j 
7 tu Oo . ° 
Gy, = Syptsgpt c+ +a, r= J. 


1T. Banachiewicz, [1]. 
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Hence we obtain the formulas for the determination of the s,;: 


————_ a1; 
$11 = Vai, ns, 


7 $n 
ix! 
J it, Ci; 4i; 2 SuiS1j 
(3) 5; =Aa@,;- > si, i> 4; Siz = = » j>i; 
I Sii 


5; = 0, t>y 


Furthermore, the solution of the system reduces to the solution of 
two triangular systems. Indeed, the equation 


AX =F, 
is equivalent to the two equations 
SK=F, SX=K. 


The elements of the vector X are determined by recurrence formulas 
analogous to the formulas for s;;, to wit: 


i-] 
— 5):k& 

eae | ee Ai Pa ee 
(4) k= Ar k= =a i>. 
The final solution is found by the formulas 


n 
kj- > Siu% 
(5) eae ee x= Et; Ci«n. 
San Sii 
In the scheme the usual control is to be employed, wherewith in 
forming the check elements we involve all the elements of the 
matrix. The control equation is an analog of that of the compact 


scheme: 
a 
Ei = Dszth;. 
reat) 
With the square-root method, one has to record only the approxi- 
2 
mately 5 elements of the matrix S and the 22 components of the 


vectors K and X. 
We exhibit in Table VI the solution of our system by the square- 
root method. 


TasB_e VI. The Square-Root Method 


0.66 
0.44 
0.22 
1.00 


054° | 0.66 
0.10270 0.17939 
0.83537* | —0.18533* 


0.70560 


— 1.25778 0.04348 1.03917 1.48238 


—0.25779 1.04349* 2.03917 2.48238 





Horizontal sums in superior position. 


0.3 
0.5 
0.7 
0.9 
0.3 
0.41211 
0.59336 
1.04597 


© Obtained by rounding radicand to five figures before extracting root, or by rounding numerator before division. 


2.92 

1.60173 
1.24340 
1.75157 


(2) 
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The computation of the elements 5;; (as also the elements &; and 
k;) is carried out by rows in sequence. Any diagonal element is 
calculated as the square root of the difference between the cor- 
responding element @ and the sum of the squares of all previously 
computed elements s located in the same column. A nondiagonal 
clement s;; is obtained by subtracting from the element @,; the sum 
of the products, row by row, of corresponding elements s taken 
from the columns with numbers i and j. This difference, once 
obtained, is divided by the diagonal! element of the row. 

Thus 


434 — 513514 — S2a5a4 
533 \ 


0.22 — 0.54 -0.66 —0.10270-0.17939 
0.83537 = —0.18533. 


The return course is carried through by the analogous formulas (5). 

With a little experience, the computations, by formulas (3), (4) 
and (5), will go easily. 

In case the elements of the matrix are such that radicands of the 
expression for s;; are negative, one will encounter no substantial 
difficulties while using the square-root method. Indeed, in this 
case, in the row for which s?,<0, only pure imaginary numbers 
figure; operations upon them are nowhit more complicated than 
upon real numbers. 

Let us clarify this statement with an example 


534 = 





1.41421 —0.70711 0.70711 2.82843 4.24265 


1seli4i | -2.21360; | —4.427191 | —5.05965; 
2.32379 5.93858 8.26237 
112 077779 | 255556 | | 
2.01112 1.77779 3.55556 
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The square-root method is now widely employed where the solu- 
tion of symmetric systems is called for; it can be recommended to the 
reader as one of the most efficient methods. 


§11. THE INVERSION OF A MATRIX 


As has already been remarked in the introduction, the problem 
of solving a nonhomogencous linear system and that of inverting a 
matrix are closely connected with each other. 
Indeed, if for the matrix A its-inversc, A-!, is known, then on 
multiplying the equation 


(1) AX =F 
on the left by .-1-!, we obtain 
(2) X = A-'F, 


Conversely, the determination of the clements of the inverse 
matrix may be reduced to the solution of 2 systems of the form 


(3) 


LIM? 


; Gita = 63; 
= Vina th 
where 06;; is Kronecker’s symbol. 
The last follows from the definition of the inverse matrix: 


AA-1 =] 


and the rule for matrix multiplication. 

A technique employed in structural mechanics for determining 
the solution of a system by means of the so-called influence 
numbers! is none other than the solution of the system by construc- 
tion of the inverse matrix. The influence numbers themselves are 
the clements of the inverse matrix. 

The solution of the # systems necessary for the determination of 
the x2 elements of the inverse matrix we shall perform by the scheme 
for several equations, as shown in Table VII. As the result we will 
obtain a matrix consisting of the rows of the inverse matrix, arranged 


VAL A. Umansky [1]. 
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in counter-order. For control of the computation and an estimate 
of the accuracy of the result, the multiplication of A by A-! is 
expedient. 

The theorem concerning the factorization of a matrix into the 
product of two triangular matrices makes possible the construction 
of a compact scheme for computing the elements of the inverse 
matrix.! The scheme requires 2n? entries in all, of which n? entries 
give the elements of the inverse matrix. 

Let 


(4) A=CB, 


where the elements of the triangular matrices C and B are deter- 
mined by formulas (2) § 8, which we reproduce here: 


jal Des. “Gs 
Cp = a p> Cubij 12 ds 


iv! 
a;;— > iby; 
b; =——__, i <j 63; =I, 
Let us denote the elements of the inverse matrix A-1=D by d;,;. 
We have, obviously, 
(5) D = B-1C-1, 


We shall show that the elements d;; can be determined without 
inverting the matrices B and C. 
Multiplying equation (5) by C on the right, we obtain 


(6) DC = B-}, 


The matrix B-! is obviously also triangular, with units along the 


principal diagonal. We know, therefore, moth of its elements: 


n(n—1) 
2 


will be null and the remaining x will be units. 
Multiplying equation (5) by B on the left, we obtain, analogously, 


(7) BD =C-). 
1 F, Waugh and P. Dwyer [1]. 
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n(n—1) 


5) of its elements will be 


Since the matrix C-) is triangular, 





zeros. 
It is readily seen that the system obtained by combining the 
n F . 
ner) equations of the system DC = B-! that were mentioned above 


{n—1) 


: n : : 
with the — 7 equations of the system BD=C-! is a recurrent 


system that makes possible the determination of the n? elements of 
the inverse matrix. 
We give it in extenso for n=4: 


€114j1 + Cardia +eaidigteaidis = 1 0 


oO fo ww 


Cood jo + Cyod;3 +es2d;4 = | 


oo a > <> 


Cyad;g+eqadj4 = I 


C4sd;4 = 


4 
dy, + byad2; + byada;+b14d4; = 0 
dy + bogd3;+ body; = 0 0 
d3;+b3qd4; = 0 


From the equations of the first group for i=4, dyy, dy3, dao, dg, are 
determined in succession. Next d34, das, d)4 are determined from 
the cquations of the second group for j=4. Following this the 
process goes on analogously and we use the formulas of the first and 
second groups in turn. Explicitly, d3;, dg2 and d33 are determined 
from the equations of the first group for i=3, and ds, and d)3 are 
determined from the equations of the second group for j=3; dy, 
and dy». are determined from the equations of the first group for 
i=2, and dj2 from those of the second group for j=2; finally, d,, is 
determined from the equations of the first group with i=1. The 
inversion of the matrix by the compact scheme is exhibited in 
Table VIII. 


Tasre VII. 





0.09320 
0.70840 
— 0.13640 
0.11316 
0.69785 
— 0.15482 
l 


0.66 


0.22 
1.00 
0.66 
0.16280 
— 0.13640 
0.56440 
0.19767 
0.15482 
0.53222 
0.22185 


0.49707 


— 0,49247 
~ 0.57698 
— 0.70570 


— 0.68624 


— 1.37835 
—LOLl49 
—0,12304 

2.50759 


1.21448 
0.11316 
0.19767 
— 0.16216 


— 0.22278 


— 0.44747 
— 0.26143 

1.33221 
=—0,12305 


Inversion of a Matrix: Single-Division Scheme 


0 
0 
1 
9 
0 
0 
1 
« 
0 
1 
0 
1, 


43297 
0.22185 


0.44560 
1.53183 
— 0.26142 
- Lona 


z 
3.62 
3.18 
3.08 
3.32 
3.62 
1.65960 
1.12520 
0.93080 
2.01508 
0.93740 
0.60275 


6) 
1.34307” 
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0.09320 
0.16280 


TaB_e VIII. Compact Scheme for Inverting a Matrix 


0.54 
0.32 


1.00 
0.22 
0.54 
0.11316 
0.69785 
—0.15482 


0.66 
0.44 


0.22 
1.00 


0.19767 ~0.12304 
| -0.22185¢ | ~ 1.01148 
| 0.49787) 1; — 1.37834 


| 0.66 2.50759 


1.33221 


. D- 
— 0.26143 


— 0.44745 


—0.12304 


| 


— 1.01149 
—0.26142 


- 
A 1.53183 


0.44560 


cata 


— 1.37834 
— 0.44746 
0.44560 
2.00856 
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§ 12. THE PROBLEM OF ELIMINATION 


This problem, in the simplest case, consists of the computation 
of the values of a linear form 


(1) C4Xy + CaXe+ °° +O,Xqy 


where ¢;, ¢,..., ¢, are given numbers and x), Xs, ..., +, is the solu- 
tion of the system 


Q41X) +4y9%0+ +++ +44,%, = by 


(2) Gy 1%) + Ago%e+ +++ +49,%, = be 


SD 


Gy 1X1 + agoXet + A AnaXy = b> 
whose determinant is different from zero. 

The natural procedure for solving this problem consists in deter- 
mining the numbers x), %2,..., x, in explicit form and substituting 
them in expression (1). This may be avoided, however, as follows. 

Let us tabulate the matrix of the coefficients of system (2), 
adjoining the column of constant terms to the right of it, and under- 
neath it a row consisting of the coefficients of the linear form to be 
computed, the signs of the coefficients being reversed. We shall 
put O in the lower right corner. We shall have the scheme 














41) 4yy eee Ay by 

Qo; aoe ax, by 
(3) Pr 

Gn1 Ong sss Aa b, 

—C, —Ca oe. =e, 0 


or, in abbreviated notation, 


(3" = 
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Let 7}, Yo, ---, ¥, designate the solution of the system of equa- 
tions 
QyYitaayot oo * Fann = 61 
ajo + doopot +°* +4,0 = a 
(4) 1271 2272 nen 2 


| 


yn Yit dog Vot ° ++ +OnnYn = Cn 


Then 
biyitoayot +++ +b, = 


= (ayyeyt ++ $444%,) 71 


+ (doyxXy + +++ +dan%,) Ye 


+ (@gXy+ 26. + aXe) Yn 
= (aynyit ss: + 4g Yn) ¥1 
+ (ayo7, + +++ +Gy2Y,)X2 


+ (41,71 ss + Gan Yn) Xn 
= €1X1 HloXeot+ i505 7% +6,%,- 


Thus the computation of the form ¢)x, +¢oxa+ +--+ +¢,x, may be 
replaced by the computation of the form 6,7, +beyat +++ +b,y,- 
On the other hand it is obvious that if we multiply the first row of 
scheme (3) by y,, the second by ye,..., the nth by y,, and add to 
the last row, we will obtain a row whose elements situated beyond 
the double line are equal to zero; the element in the lower right 
corner obviously equals the number that is sought: the value of the 
linear form. Conversely, if we somehow choose a linear combina- 
tion of n rows such that the addition of it to the last row gives a null 
row (up to the line), the coefficients of this combination will be 
solutions of system (4), and accordingly the element in the lower 
right corner will equal the sought number. This follows from the 
uniqueness of the solution of system (4). There is thus no need to 
find the numbers 7), y2,-.-., ¥,- It is merely necessary to “annul 
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the last row”? by the addition of a suitable linear combination of the 
first n rows. This can be done by the ordinary forward course of 
the Gauss process, applicd to scheme (3). 

Just such a device may obviously be employed for the computation 
of the non-homogencous linear form ¢,x, + «++ +¢,x,+4. There will 
be merely this difference, that in the lower right corner one must enter 
not 0 but the constant d, so that the initial scheme has the form 




















Qi, AyQ we Ay b, 
(3”) 

Qny Ange ee Ong 6, 

—l; —lg... —¢, || ad. 


The solution of system (2) also may be found by this method, 
without employing a return course. Indeed, the expressions 
Xy, %2,--..,%X, are particular cases of linear form (1), with coeffi- 
cients (1,0,...,0), (0, 1,...,0),...,(0,0,..., 1). Their 
determination may be effected simultaneously by the elimination 
scheme by simultancously entering in the lower left corner the rows 
(-—1,0,..., 0), (0, —1,...,0),..., (0,0,..., —1), which col- 
lectively constitute the matrix —J. In the lower right corner, 
rather than the number 0, we place a null column. 

The source scheme for the solution of the system thus has the form 


(5 — 








Having obtained a null matrix in the lower left corner of the scheme 
by the addition of suitable linear combinations of the first n rows, 
we will have in the lower right corner a column of the values of 
the unknowns. 

The inversion of a matrix is equivalent, as we have seen, to the 
solution of n systems of a special form, whose constant terms form the 
unit matrix. Their collective solution may be effected by means of 
the scheme 

A 


SH 
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where J, as before, designates the unit matrix, and in the lower right 
corner is arrayed a null matrix of the nth order. After annulling 
all the rows in the lower left corner, by the addition of suitable 
linear combinations of the first n rows, we will obtain the matrix A-! 
in the lower right corner. 

The result of the elimination may be cast in matrix form. This 
makes it possible to obtain some generalizations. To wit: the solu- 
tion of the system, x,,..., x,, may be written in matrix form as the 
column 


= A-1), 


and the value of the linear form ¢)x,+¢.x.+ -++ +¢,x, as the 
number! cA-8. 

Such a representation points the way to the computation of the 
more complicated expression CA-'!B, where C and 8B are certain 
rectangular matrices, of which C consists of » columns and B of n 
rows, and the number of rows of C and columns of B is a matter of 
indifference. Indeed, the element of the ith row and Ath column 
of the matrix CA~1B is the number ¢,4~14,, where ¢; is the ith row of 
the matrix C, and 6, is the Ath column of the matrix B. 

The computation of the elements of the matrix CA-1B may there- 
fore be carried out by the method of elimination, applied to the 
scheme 

| B 
0 


A 
(7) soe 
-C 





After annulling the elements located in the lower left corner, by 
adding a linear combination of the first n rows, we obtain in the 
lower right corner the matrix CA-1B. 


V [Ie is to be remarked that the author's notation regards the row position as 
normal for a premultiplying vector, and the column position as normal for a post- 
multiplying one, no further diacritical marks thus being called for.] 
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Putting C= J, we obtain the scheme 
A 


(8) az 
-! 


-B 


0 








for the computation of the product A-1B. 

By transposing the scheme, one can obtain a modification of the 
method of elimination, to wit: if D=CA-1B, then D’=B'(A’)-1C"; 
hence it follows that D’ (and accordingly, D as well) may be com- 
puted by a forward course applied to the scheme 

A’ 
(9) == 








In particular, for computing the value of the linear form 
64%) + +++ +¢,x, one may also use the scheme 


A’ 








where 4’ is the matrix that is the transpose of that of the coefficients 
of the system, ¢’ is the column composed of the coefficients of the 
linear form being computed, 0’ is the row of constant terms, to be 
taken with signs reversed. The validity of this construction may 
be readily proved directly, without relying on the results set forth 
above, for if we multiply the first 2 rows of scheme (9) by x),... 5%, 
and add to the last, we will obtain zero in the lower left corner and 
€)X;+ +++ +¢,x, in the lower right corner. 

Analogously, for solving the system AX =6 we may use the scheme 


Y 


We exhibit in Table IX the solution of the system, carried out by 
this modification of the method of elimination. 

As usual, the last column of Table [X is the control; throughout 
the duration of the process cach number obtained in it must coincide 
with the sum of the preceding numbers of the same row. 


A’ 





(10) 








-’ 


Taste IX. Solution of a System of Linear Equations by the Method of Elimination 


* 
0.42 
1.00 
0.32 

0.44 

-0.5 
0.42 
0.82360 
0.09320 
0.16280 

— 0.37400 
1 


0.09320 
0.70840 
— 0.13640 
— 0.53800 
0.11316 
0.69785 
0.15482 
— 0.49568 
1 


* 
0.66 
0.44 
0.22 
1.00 
-0.9 
0.66 
0.16280 
— 0.13640 
0.56440 
— 0.70200 
0.19767 
0.15482 
0.53222 
— 0.62807 
0.22185 


0.49787 
— 0.73804 
| 


0.49247 
— 0.57698 

0.10927 
— 0.70570 


ora oo RF Oo 


see ceineall 


0.01316 
—0,19767 

0.45410 
— 0.16216 


- 0.22278 
0.37372 
0.44747 
0.04347 


0 
0 
a 
1 

0 
0 
1, 


43297 


0.22185 
0.71029 
0.44560 
1.03916 


eer Of|SO OO =- @ S}e]/o -~ © oF cS 


1.48240 


1.65960 
1.22520 
0.93080 
— 1.31400 


0.93740 


1.30727 
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qv 


. 
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Tasre X. Computation of a Product A-1B 









o 


-0.90 
0.60 
—0.60 —0.80 
ose 6 | o6e,)SC]lctt!!™CdTCtté‘iLCW:;‘;C;*C*CY 
0.16280 | —0.42 
—0.13640 | —0.54 
0.56440 | —0.66 
— 0.68500 0.25 
—0.70200, | 0.30 
—0.36900 | —0.50100 0.15 
~0.49200 | —0.66800 0.20 
1 | 0.11316 =| 0.19767 | —0.50996 


oo CO lUlOIOolUlUCUCOUlU 












—) 
i} 





——_—— J 









0.09320 
0.70840 
— 0.13640 
—0.51500 
— 0.53800 


1.65960 
1.12520 
0.93080 
— 1.29500 
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0.69875 
— 0.15482 
— 0.47596 
— 0.49568 
—0.34218 
— 0.45624 


————| 


0.53222 
— 0.61680 
— 0.62807 
— 0.45415 
—0.60554 
— 0.22185 

0.49787 
— 0.72239 
—0.73804 
— 0.53006 
— 0.70676 


— 0.15482 





— 0.49247 
—0.57698 
0.07406 
0.10927 
0.02914 
0.03885 
—0.70570 


~ 0.68624 
— 0.26182 
— 0.24053 
— 0.21234 
—0.28312 
~— 1.37835 


— 1.25753 
— 1.25781 
— 0.94295 
— 1.25728 


—0.11316 


— 0.19767 
0.41889 
0.45410 
0.28776 
0.38368 

—0.16216 

— 0.22278 
0.34171 
0.37372 
0.23227 
0.30970 

— 0.44747 


0.01846 
0.04347 
— 0.00492 
— 0.00655 





J 
0 
0 
0 
0 
0 


1.43297 
0.22185 
0.68204 
0.71029 
0.49033 
0.65378 
0.44560 


1.00394 
1.03916 
0.72652 
0.9687! 


(A-'By’ 

















0.93740 
0.60275 


— 0.47943 
(5) 
— 0.63924 
1.34327 
0.81072 
0.03954 






1.45096 
1.48240 









1.06466 
1.41957 
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Observation. In solving systems by the method of elimination 
the number of operations somewhat exceeds that for the Gauss 
method. The uniformity of the process, however, and especially 
the absence of a return course, not infrequently make this method 
the more convenient one. 

For inverting a matrix the scheme 


Aoyod 
(ity Sl 





-1 || o 


may be used. Here, after completion of the process, we will obtain 
in the lower right corner a matrix that is the transpose of A-!. 

Thus for each of the problems analysed we have two schemes, 
defined cither by the matrix A or by its transpose. We note that it 
is most efficient to use that scheme which contains the least number 
of rows in the lower left corner. Thus in the problem of solving a 
linear system without a return course it is most efficient to use the 
scheme with the transposed matrix; in the problem of computing 
the product A-!B—the scheme with the transposed matrix in case 
the number of columns in B is less than n; the given matrix should 
be used as such if the number of columns in B is greater than n. 

In Table X we exhibit the computation of the product A-!B. 
Here, as A, we have taken the matrix 


1.00 0.42 0.54 0.66 
0.42 1.00 0.32 0.44 
0.54 0.32 1.00 0.22 
0.66 0.44 0.22 1.00 


and as B, the matrix 


0.25 0.30 0.15 0.20 
0.45 0.50 0.30 0.40 
0.65 0.70 0.45 0.60 
0.85 0.90 0.60 0.80 
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The computation of A-!B has been effected by scheme (9) with 
C’=I, We have as the result of the computations 


— 1.25753 -—1.25781 -—0.94295  — 1.25728 
0.01846 0.04347. —0.00492 -—0.00655 
1.00394 1.03916 0.72652 0.96871 
1.45096 1.48240 1.06466 1.41957 


AB = 


§ 13. CORRECTION OF THE ELEMENTS OF 
THE INVERSE MATRIX 


Inversion of a matrix by any of the schemes exhibited above 
offers no certainty as regards the accuracy of the results obtained, 
owing to the inevitable rounding errors, the influence of which on the 
final result is hard to gauge. For a check of the accuracy of the 
matrix Dy obtained from the given matrix «A by whichever process 
of inversion, one should determine the product ADg; the discrepancy 
between it and the unit matrix will indicate the degree of inaccuracy 
in the results obtained. 

Let this check computation reveal that the approximation Dg to 
al-) is such that j'Foli<k <1, where 


(1) Fy = [-ADo. 


The first or second norm of those introduced in § 5 is conveniently 
taken as the norm of this matrix, they being the most readily 
computed. 

Under this condition the elements of the inverse matrix, A-!, may 
be computed to as high an accuracy as is desired by means of the 
following iterative process.} 


'H. Hotelling, (1). We note that Hotelling took as norm the expression 


{ 
; 2 ; (ay)? » for which the requirements of § 5 are not satisfied. In § 17 the 


relation between this expression and the third norm is explained. 
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Let us form the sequence of matrices 

D, = Do(l+Fo), F, = 1- AD, 

Dz, = D,(I+F,), Fo = 1-ADo 


rr Py 


Da = DA +F, -1)s F, = I—AD,,. 


(2) 


We shall show that the matrix F,,=/—AD,, is equal to F2". 
Indeed, 
F,, = 1-AD,, = I— AD,_\(1+Fn_3) 


(3) = 1-(1-F,-1)(1+Fa-1) 

= Fi1= Faas = Fy’. 
Hence it follows that 
(4) D,, = A\(1-F2"). 


The last formula shows that D,, approaches A-!, the convergence of 
the process being very rapid. Let us make an estimate of the 
error, taking into consideration the fact that A-!=D)(ADp)-! 
=D)(I-f)-!: 


[Dy —A-H] = ||] — 4-43") = || — DoE B)-4F I 
aa ken 
(5) < |\Doll 7) FR" < Doll -— 


It is seen from this estimate that, if only the iritial approximation 
be so chosen that ||/— ADo||<k <1, the number of correct figures of 
decimals increases in geometrical progression. 

The successive approximations should be computed by lifting the 
parentheses in formulas (2), viz.: 


(6) Dy = Dy +Fm-1) = Da-1+ Da-1(1— ADg-1)- 


The second summand will here play the role of a small correction 
to the first. 

Sometimes, starting with the matrix Fo, it is expedient to form the 
matrices F, =F, F, =F§=(F,)*, squaring the matrix successively, 
afterwards utilizing formula (6). 
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Observation. In case A is a symmetric matrix, and one has taken 
as the initial approximation Dp a symmetric matrix, all the subse- 
quent approximations will likewise be symmetric matrices, although 
the matrix Fy may turn out to be non-symmetric. Indeed, from 
formula (6) it follows that D,=2D,,_,—D,-;4D,,_;, whence, on 
assuming that A’=4, D’_,=D,_1, we obtain 

Di, = 2Din-1 —(D,-14AD,—1)' = 2Din-1 —DaaA' Da... 
= 2D,,-1 —- Da- AD, = D,,- 
As an example, let us invert the matrix 
1.00 0.42 0.54 0.66 
0.42 1.00 0.32 0.44 
0.54 0.32 1.00 0.22 
0.66 0.44 0.22 1.00 


As the initial approximation let us take from Table VI, § 11, the 
result of the inversion of the matrix A by the single-division method 
(retaining four figures after the decimal) : 


2.5076 -—0.1230 -—1.0115 —1.3783 
— 0.1230 1.3322 —0.2614 —0.4475 
—1.0115 -0.2614 1.5318 0.4456 
—1.3783 —0.4475 0.4456 2.0086 
The check computation gives, for Fo, the following value: 
—52 -18 20 —-50 
—60 8 —-10 10 
-18 -3+ 26 -10 
—-66 20 10 —5+4 


Hence we see that Fo i!;<0.000150, {{yl|;,;< 0.000196. For the 
estimate of the error Ict us take ‘lF9),. On the strength of formula 
(5) we have, taking into account that ||Dol|, <5: 


. (0.00015)2 


~— Ac ae 
| D,-aA Ih <5 =F po0r5 <0:0000001. 
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Thus D, gives for A-! a value correct at least to a unit of the 
seventh decimal place for each of its elements. 
Doing the computation, we obtain 
,2.50758616 —0.12303930 —1.01148870 — 1.37834207 
—0.12303930 1.33221281 —0.26142705 —0.44745375 
— 1.01148870 —0.26142705 1.53182667 0.44560858 
— 1.37834207 —0.44745375 0.44560858  2.00855152 


The check computation /— AD, gives 


20 0 1 

1o-1 1 
I—AD, = 10-8- 

10 oO 0 

10 oO -1 


§ 14. THE INVERSION OF A MATRIX BY 
PARTITIONING 


It is sometimes expedient to partition a matrix beforeinversion. 
Let us examine the formulas for inverting a matrix of the nth order 
partitioned into four cells by the scheme 


A | B 
5=|(——-_], 
c |p 
where A and D are square matrices of orders p and g; p+q=n. 
Let us seck the inverse matrix also in the form of a cellular 


matrix: 
(— | L 
s-! = | ————-}, 
M | 7 


K and N being again square matrices of orders p and ¢. 
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In conformity with the rule for multiplying partitioned matrices, 
the following matrix equations must hold: 


AK+BM = 1 


AL+BN =0 
CK+DM = 0 
CL+DN =1. 


Multiplying the third equation on the Icft by BD-! and sub- 

tracting it from the first, we obtain 

(A-BD-IC)K = ], 
whence 

K = (A-BD-'C)-!. 
Furthermore, from the third equation we find that 

M = —D-'CK. 

In like manner we find from the second and fourth equations that 


N = (D-CA-!B)-! 
and 
L= —-A BN, 


Of course these formulas have been derived on the assumption that 
all the indicated matrix inversions are realizable. 

The inversion of a matrix of order » thus reduces to the inversion 
of four matrices, of which two are of order p and two are of order q, 
and to several matrix multiplications. 

The formulas developed above can be altered so that for the 
computation of the matrices K, L, Af and N only two matrices, of 
orders p and q, need be inverted. For, as can readily be verified, 


N = (D-CA-'B)-1,) Mo = —NCA-), 

L= -A'BN, K = .l-! -A-'BM, 
and, analogously, 

K = (A-—BD-'C)-1,, L = —KBD-}, 

M = —D-'CK, N= D-!—D-\CL. 
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The last formulas show that the method of partitioning is con- 
veniently employed when any diagonal cell is easily inverted. 
Let us take as an example the inversion of the matrix 


1.00 0.42 : 0.54 0.66 
0.42 1.00 : 0.32 0.44 


Pererererererrrrretrrerce rr err errere 


0.54 0.32 1.00 0.22 
0.66 0.44 : 0.22 1.00 


The computation will be performed as follows: 
1) We compute the matrix 4-}: 


ene 1.21418 ea) 
~ \ -0,50996 —-1.21418 
and form the products 
0.49247 0.57698) 4 _ ee 0.11316 
0.11316 0.19767 }’ ~ \ 0.57698 ee | 


0.30215 0.37482 
0.37482 0.46778 } ° 


AB = ( 


CA-!B = ( 


By twice computing the last matrix, as C(A-!B) and (CA-!)B, we 
obtain a check on the preceding computations. 
2) We form the matrix 


69785 —0. 
(D-CA-1B) = ( 0.6978 Goes) 


— 0.15482 0.53222 


and find its inverse 
1.53183 0. 
N = (D-CA-!B)-1 = ( eta : 


0.44560 2.00855 
3) We compute the matrices 


—1.01148 -—0.26142 
M = —NCA-! = ( — 1.37834 —0.44745 


-I. — 1.37834 
= -47BN = ( 1.01148 1.37 ) 


—0.26142 —0.44745 
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and 


30 — 0.12305 
K = A-!—A71BM = ( aa . ). 


— 0.12305 1.33221 
The sought inverse matrix will thus be: 


2.50757. —0.12305 -—1.01148 —1.37834 
— 0.12305 1.33221 -—0.26142 —0.44745 
—1.01148 —0.26142 1.53183 0.44560 
— 1.37834 —0.44745 0.44560 2.00855 


S-1 = 


§15. THE BORDERING METHOD 


In this section we shall consider computational schemes for the 
inversion of a matrix and the solution of a linear system that are 
based on the idea of bordering. 

The given matrix A will be regarded as the result of bordering a 
matrix of the (n—1)th order, the inverse matrix of which will be 
considered to be known. Thus lect 


4) a2 4, a-1 4}, 

42) azo 49, a-1 ao, 
(1) 4, = 

ay-1, 1 4,-1, 2 4-1, n-1 Gn-1,n 

Qn) 4,2 an, n-l an 


Here A,_, denotes the aforementioned matrix of the (a—1)th 
order, 


4}, 
(2) = (4,1, cee Ay, n-1)> u, = ( : . 


Gn-1, 1 
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We seek the matrix A-! as a bordered matrix too, in the form 


Pi-1 vn 
(3) D, = A; = ] ) 
Gn a 


: : I 
where P,_, is a matrix, g, a row, r, a column and a number, all 
nA 


of which we have to determine. 
By the rule for multiplication of bordered matrices we have 


A,-1 uy Pt YT, 
AA-} 1 
Up, Gon Gn a, 
un 
A,-1P, -1 Fn Ins Ayn to I 07 
UP a1 + nn Jas Ugly tet 01 
Hence 
(4) An-Pa-1 +4,9, =I 
(5) Up? a1 +4anIn = 0 
uy 
(6) Anat =0 
a, 
(7) at = 


We have from equation (6) 

1 
(8) ,=-— 
Substituting this value of r, in (7), we obtain 
(9) Gy = O,,—U,A, >t, 
Furthermore, we have from (4) 


(10) Pit = Ay, — Art In 
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and therefore, on the basis of (5) and (10): 
vA, fe 1 v, A, 2 14n9n Be Fandn 


oe vg! — (4aa— 0) Int Fan Gn = AS +29, = 0. 


Hence 
vAD 
(11) m= Et. 
Finally, 
(12) P.,= Act, + fale, 
. a Pa 
n 
Thus, at last, 
An} 4 Anata, An, 
n-} “a, Op 
(13) Al) = ‘ 
_ ale, l 
a, a, 


where @, =p, —UAg ta 

The constructed formula is obviously a particular case of the 
formulas for inverting a matrix by the partitioning method, # here 
equalling n—1 and q being |. 

The method developed here is basically a method of inverting a 
matrix by successive borderings: one constructs in succession the 
matrices inverse to the matrices 


; Q11 G2 4: 
hag dee a 4120 413 
(ay), A ’ CDS oe a 


.) Aao | 
OD oer 43, Ayn ay 


of which each successive one is obtained from the preceding by 
bordering. Each step of this process is accomplished on the basis of 
formula (13), viz.: if Az_', is already known, the following operations 
must be carried out in order to find Ay!: 

1) The computation of the column:—A,!\u,. The elements of 
the column, fj, ..- B,-1, »» are found by accumulation. 

2) The computation of the row: —v,471, with clements 
Vals: Ya, a-b 


108 Systems of Linear Equations 


3) The computation of the number 
n-1 a—! 
O, = Any t 2, QniBin = Ang t 2 Ginnie 


(The computation twice of the number a, is a good check on the 
preceding computations.) 

4) Lastly, the determination of the elements d,, of the inverse 
matrix by the formulas 


diy = dyp+2ebee (i, k<n—1) 
(14) 


d, = $8, dys Pat (iken—l), day = = 
Here dj, are the clements of the matrix A7_,. 

In case of a symmetric matrix the scheme is obviously abridged 
by exactly onc half. 

In Table XI the inversion of a matrix by the method of successive 
bordering is exhibited. 

Each step of the process is set up on the following scheme: 


-! -1 
Ay | uy, — Ayu, 


(15) . 





R 


-1 
—vAT 


The method of bordering may also be useful in application to the 
solution of a system of linear equations. It is especially convenient 
to utilize this method in case it is necessary to solve a system for 
which the truncated system—obhtained by striking out one equation 
and one unknown—has been solved earlier. Such a situation is 
often encountered in applications. For example, in solving prob- 
lems of mathematical physics by the method of B. G. Galerkin or 
Ritz, it may occur that the solution obtained as the result of utilizing 
the (n—1)th coordinate function is insufficiently exact; if the 
addition of just one more coordinate function is sufficient for the 
construction of a more exact solution, the new system for the deter- 
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TasLe XI. Inversion of a Matrix by Bordering 














Scheme: 

1.00 0.42 0.54 0.66 Ay. 7 ttn a1 t= (Bin) 
0.42 1.00 0.32 0.44 tn nn 

| 

—tady 1= (Ym) on=4na— end ain 
1.00 0.42 —0.42 i F cirainrlad? : : 
nt 
0.42 1.00 n= dont S ~ autBin=@nn + *s Tins nt 
—0.42 0.82360 Ag?) = Da-1= (di) 


- Ay'=Da=[dul 











1.21418 —0.50996 | 0.54 | —0.49247 i 

—0.50996 1.21418 | 0.32 Se oe = 
0.54 0.32 1.00 dig = Be dua (ikea 1); 
0.49247 —0.11316 0.69786 an = = 

~“Tse171 0.43010 —0.70509 | 066 |-oeszs || 
0.43010 1.23253 —0,16215 | 0.44 = 0.22277 

—0.70569 —0.16215 1.43295 | 0.22 0.22185 

0.66 O44 0.22 1.00 

0.68623 —0.22277 0.22185 0.49788 | 

9.50754 —0.12306 —1.01147 —1.37830 


— 0.12306 1.33221 —0.26141 —0.447-44 
1 

— 1.01147 ~o261414 1.53180 0.44559 

— 1.37830 = —0.44744 0.44559 2.00852 
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mination of the coefficients is obtained by bordering the previous 
system. 

The method of bordering is applied to the solution of a system of 
linear equations in the following manner. 

Let the system have the form 


(16) A,X, = F,. 
Let us employ the symbols 


u, ie 
i) 42s ae ae <" 
Up Gan de X, 


Then 
[ 0 es 
0 } Sn 


me (gears — Au, i 
Qa, -—v An! 1 Si 


a’a-l 
( Ant Foy I ( Ay ty Ag i Fn—1 —AD tt, 
0 —vATF,1 +f, 


os 
fl 
G——, 
* S&S 
— 
Ul 


a, 


But A,_’,F,-; is the solution of the truncated system, i.e., the system 
QyXyt oes $41, , ha HS 
4-1, 41+ mene +4,.1, n—1*p-1 =Sfi-1 


which solution we shall denote by X,_1. 
Analogously A7!,u, is the solution of that same system with the 
constant terms changed, that is, the solution of the system 


4,1, wit: +4,_1, n-1%, -1+4,-3, 2 0, 


which solution we shall call Q,_:. 
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Knowing X,_, and Q,_1, we easily compute X,, for 


te l ( —Q,-1X, -1 ea 
A= +—_—_—_—_ 
0 Gan +UgQn—1 —0,X,-1thn 


= X,-1 ip ek eee Qn-1 
0 Gin +YnQn-1 1 : 
Thus for the determination of X, we must, in addition to X,_, still 
compute Q,_}.- 

If the truncated system had been solved by Gauss’s method, it 
would be necessary to have added yet another column in order to 
have found Q,_,. The computation of the forward course is 
effected only for this column, utilizing the left half of the scheme, 


already known. Afterwards Q,_, is determined by the usual 
return course. 


§ 16. THE ESCALATOR METHOD 


In inverting a matrix by the bordering method, the essential 
role in passing from the inversion of the matrix A,_; to the inversion 
of the bordered matrix A, is played by the computation of the expres- 
sions —A,_',u, and ~—v,A;_},. 

It is obvious that the components of the vector — Aj_',u, are none 
other than the solution of the system of equations: 


(0) QZ tAyozogt os $41, g-1y-1, et ay = 0 
Mga, Zap F Og-1, Ze oot Aga, p12 4-1, AA MHI, 4 = 002 


1 [-Ay yu, of §15 is a column vector with (k—1) components, say =,, where 
ro et) ees 1, x} and thus 


—Apl iy = 2 
m= Aya My 
ie., Ay. oy = 0. (101) 
But y= {ay, oe iy, x}: 


(101) is therefore the author’s (0)=(2) in matrix notation. Note that when 
k=n+1, (101) becomes the author's (1), with 2, ..)=%;-] 
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Analogously, the components of —v,4j_1, form the solution of the 
transposed system. 

The successive solution of such systems, for k=2,...,2,2+1, 
lies at the basis of the so-called escalator method for the solution of 
of systems of linear equations.!_ By the same token, the escalator 
methad is closely connected with the method of bordering. 

The author of the method limits himself to imparting a recipe for 
constructing the solution, and to formulas for the case x =4. 

The author considers the essential merit of the method to lie in 
the existence of a reliable check, by means of which the accuracy of 
the computations can be regulated; in particular, the method gives 
a result that is sufficiently reliable even in case the determinant of 
the coefficients of the system is small. 

We shall set forth the escalator method only in application to 
systems with symmetric matrices, giving the computational scheme 
the compact form. The connection between the escalator method 
and Gauss’s method will thereupon be revealed. 

So let there be given a system of equations 


443% F4yo%ot 62+ $4 +41, a1 = O 
(1) Gah dah oh th geek ea Re ak GR ear 


441%, +4,9%o+ >>> +4,,%, + 4p, at] = 0, 


the matrix of the coefficients of which is symmetric. 
Designate by 21,,... Z,-1, ¢ the solution of the system: 


Oy Zag os $y, 412-1, 4+ 4 = 0 
(2) eee de las 9 ce he Pye, ta 


Oy_y, Zaye oe Hg, g~1Z4~1, 2+ G-1, « = O- 


The numbers z; ,41=*; obviously form the solution of system 
(1). If, therefore, we establish a means of computing in turn the 
numbers z;, for i< k<n+ 1, we shall by that fact have given a method 
for constructing the sought solution. 

Let us first assume that all the numbers z;, have already been 
computed for i<k<n. 


1 J. Morris ({}. 
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By means of them let us construct the matrix 


Ize Zig +++ Zn 
0 1 293 + + + Lan 
(3) Z _ 0 0 1 coe § Z3y . 


0 0 0... I 


It is easily seen that the matrix AZ has zeros above the principal 
diagonal. Indeed, 


(4) Cy = AZ 
413) Ay ZyQF4ya, «~~ ArpZig9H@yoZon + °° > +Ajq 


491, 421212 + 4g9, - ~~ 5 Gai 21g t+ 4egZen + °° * +42, 


Fair B_iZ12 + 4qas » + +s WqiZinFAggZeg t+ ° °° FAg, 


and, in view of the definition of the numbers z;;, all elements lying 
above the principal diagonal are equal to zero. The non-zero 
elements of the matrix C, are computed by the formulas 


(5) Cig = Gyr ZizHOjoZajy+ 06+ $4; j-12j-1, ;+4ijy TZJ. 


Hence follows the connection between this method and that of 
Gauss, when the latter is considered in the light of a resolution of the 
matrix into factors. 

Indeed, from (4) we have 


(6) A= C; 2-1, 


But the matrix Z-! is a triangular matrix with unit diagonal and 
zero elements below the diagonal; C, is a triangular matrix with 
zero elements above the diagonal. Comparing this resolution with 
the factorization of the matrix corresponding to the single-division 
scheme of the Gauss method (A =CB), we obtain, on the strength of 
the uniqueness of such a resolution, that 


(7) Z-1=B, C=C. 
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Let us represent the matrix C in the form 
| 0 0 eeu. 10 C1) 


21 1 0 sas O Cog 
(83) C= | yg, yor 1... 0 C33 = TA. 
Ya1 Yn2 YnB --* 1 Cnn 


Here 
C;; . 
a aid, > ,. 
Vii oj; J 


We shall show the matrix [‘=CA-! to be the matrix that is the 
transpose of the matrix Z-!. In doing so we shall make essential 
use of the symmetry of the matrix 4. We have : 

A=TPAZ-1= A = (Z-1)'AI". 
But (Z-!)' is a triangular matrix with zeros above the diagonal; J” 
is a triangular matrix with zeros below the diagonal and units on it. 
On the strength of the uniqueness of the factorization, I” = Z-!. 

This circumstance permits us to give recurrence formulas for the 
successive determination of the numbers z;;._ Let us assume, indeed, 
that we have already computed the clements of the first & columns 
of the matrix Z. Then by formulas (5) the first & columns of the 
matrix C can be computed, and consequently the first & columns of 
the matrix J’, that is, the first k rows of the matrix Z-!. In order 
to continue the process we must compute the elements of the 
(k+1)th column of the matrix Z. Since the diagonal clement of 
this column equals unity, and the elements below the diagonal 
equal zero, it remains for us to give formulas for the computation 
of z; 441, where i<k. From the equation ZI’ =TJ we obtain, on 
the strength of the rule for the multiplication of matrices, the follow- 
ing recurrence formulas: 


Yer, 1+ Yesr, 22124 + HY 041, 2212+ 21, e411 = 0 
(9) Verio te HY ee, 2224+ 22, a1 = 0 
Vist, ete, 41 = 9, 

defining the elements of the (k+1)th column of the matrix Z. 
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The reliable check alluded to earlier consists in the actual reduc- 
tion to zero of the subdiagonal elements of the matrix C’.! 

We subjoin the compact scheme of the escalator method for a 
system of four equations. (Recall that for uniformity of computa- 
tion the constant terms are tabulated in the left parts of the 
equations.) 


te te te 
Do 
oe 


“= 








[Since 2441=(21, n¢1-+ + Ing, nvis=%1-- + %,} = is the sought vector, the 
’ . . fi >= aD 
computation is conducted with augmented matrices A and Z as follows. Denote 
system (2) by Ax+A=0 for brevity, and form the augmented matrices 


: : val | E : onl Cc : r 
-v : ollo i a) b-wz i -ae ee | 


Here r is the vector of residuals, which must equal zero to the limit of computa- 
tional accuracy; the trailing cell of C, —A’x, is not computed, being of no utility, 
Of Cit remains true as of C that all clements above the principal diagonal are zero, 
and therefore the reasoning that led to the equations for the determination of ¢;;, 
yy and z,, remains valid. The following computor’s formulas summarize the results: 


Denoting by a, the ith row of a, 
a the jth column of a, 


we have, since C=4Z 


C= % 2G 
(10) So ¢=4( 24, for Ais symmetric. 
4 
MT al, i>j, 
(it) mae Peds 


(12) 2y is yielded by =; {)=0 for i#j.] 


Tas_e XII. 


0.54 


0.11316 
a 


I’ 
1.00 


0.000003 —_-0.000003 0.69786 


U 


0.000006 0.000005 —0.000001 


0.000010 0.000008 —0.000009 


0.66 
0.44 
0.22 
1.00 
-09 


0.66 


0.19767 


—0,22185 


1.00 


0.49788 


— 0.000001 


—0.3 


— 0.45410 


— 0.71028 


— 1.48237 


1.00 


The Escalator Method: Compact Scheme 


—0.42  —0.49247 


1 —0.11316 


— 0.68623 
—0,22277 
0.22185 
| 
0 


Computing Formulas 


— 1.25776 
0.04350 
1.03914 
1.48237 
1 


cy=atz 2G where a(; is the ith column of A. 


ey 
74 
ey 


=i is yielded by 2; 4{;=0 for i#j,where z; is thei th 
row of Z > 





iS) | 


suoyonby avaury fo suapsty 
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The scheme consists of three parts. In the first part is entered 
the matrix of the coefficients of the system (symmetric, under the 
condition). In the second part we gradually tabulate the elements 
of the columns of the matrix Z. In the lower part are entered the 
diagonal and subdiagonal elements of the matrix C’ and the ele- 
ments of the matrix J”. We note that the elements of the kth row 
of the matrix I’ are obtained by multiplying the columns of the 
matrix A by the éth column of the matrix Z and dividing the sums 
obtained by the clement ¢,,. A column of matrix Z is filled up by 
formulas (9). In Table XII an illustrative example is given. 


§17. THE METHOD OF ITERATION 


We shall pass now to a description of iterative methods of 
solving systems of linear equations. These methods give the solu- 
tion of the system in the form of the limit of a sequence of certain 
vectors, the construction of which is effected by a uniform process 
called the process of iteration. 

Let the system of linear equations be given in the following form: 


Xp = AyyXp + ayoret + Hayy thy 


(1) 


Xgqg = Aa1X}+GooXgt+ --> +d2,%,+fe 


Ky = 4g X + 4yeXet + °° t+ OygXa tty 


At first glance such a form of notation for the system may seem some- 
what artificial, if only owing to the presence of like terms. This 
form is, however, especially convenient for the application of 
iterative methods. In the following section a number of devices 
will be considered which transform the given system into form (1). 
Every such device will give some modification of the iterative 
method. 
Let us write system (1) in the form 


(2) X = AX+F, 


where A is the matrix of the coefficients, F the vector of constant 
members. 
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Let us construct the indicated sequence of vectors as follows: in 
the capacity of an initial approximation let us take a certain vector 
X(, chosen, generally speaking, quite arbitrarily. Next let us 
construct the vectors 


XW = AXYO4-F 


X® = AXW4 6 
3) 0 


se © 6 6 e 8 el 


If the sequence X(, XM, .,.,X,... has the limit X, this 
limit will be the solution of system (2), for on passing to the limit in 
the equation X) =AXt-)+F as k->oo, we obtain X=AX+F, 
which proves our statement. 

We shall now give necessary and sufficient conditions of the con- 
vergence of the process of iteration, and also a number of sufficient 
conditions and estimates of the rate of convergence, based on the 
results of § 5. 


THEOREM 1. For convergence of the process of iteration with any 
initial vector X and with any value of the vector F, it is necessary and 
Sufficient that all proper numbers of the matrix A be less than unily in modulus. 

Proof. One can readily convince oneself that 


(4) Xb) = ALX(0) 4 (14+ A+ eae + Al )F, 


Indeed, for k=1 this is true, and for the remaining & it is verifiable 
by the method of complete induction. It hence follows that for the 
convergence of the iterative process with any F it is necessary that 
the serics /+A+ +--+ +Al-14 -+- converge. Indeed let us take as 
X a null vector, and as F a vector all of whose components except 
the ith are equal to zero, the ith component being equal to unity. 
Then X® equals the ith column of the matrix 7+ A+ --- +Aé-1, 
For convergence of the sequence X“ it is necessary and sufficient 
that all elements of the ith column of the matrix J+ A+ +++ +Aé-! 
have a limit, and since i may be taken as anything from | to n, the 
scries [+ A+ --- +A4-1+4 +--+ must converge. But this condition 
is also sufficient, for with it satisfied, A‘-0, and consequently the 
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first summand, A‘X ©, in equation (4) tends to zero, and the second 
summand, ([+A+ --- +A*-1)F has a limit equal to (7+A+--- 
+At-14 .-.)F. In §5 it has been shown that the necessary and 
sufficient condition for convergence of the series /+A+ - 
+Ak-14 ... is the inequality |A,;| <1, for all proper numbers /; of 
the matrix A. This proves Theorem 1. 

Since the condition of Theorem | is difficult to verify, it is better 
to judge the convergence of the iterative process by means of some 
sufficient signs—those connected directly with the elements of the 
matrix. These criteria follow at once from Theorem 2, § 5. 


THEOREM 2. In order that the process of iteration converge, it is 
sufficient that any norm of the matrix A be less than unity. 

Proof. Indeed, if ||A[| <1, all proper numbers of the matrix are 
less than unity and therefore, on the basis of Theorem 1, the iterative 
process converges. 

We shall now give estimates of the rate of convergence of the 
iterative process in terms of the norm. Here the selection of the 
norm of vectors is a matter of complete indifference, but the matrix 
norm must be subordinate to the vector norm one has chosen. 


THEOREM 3. /f ||Al| <1, then 


WF LAL 


(5) JX — XU] < |X|] [44] +e T=|4r 


Proof. We have 
|X- X®|] 

= ||(2—A)-1F— (I+ A+ +++ + Ab!) F— ALX (|| 

< [IU A)“A- (14 A+ ++» +A) Fl + AX 
€ [F-A)-'-— Ft At +++ + At 3D] I+ AUX Ol 


a Al 
lll 
It is often important to compare the accuracy of two successive 


approximations, i.e., to compare the quantities || —X|| and 
\X— X 4-1], 


€ AlhXOl +747 
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Such a comparison may be made on the basis of the following 
theorem. 


THEOREM 4. ||X— X“||<|jAl] |X—X4-]) 
Proof. Indeed, from the equations 


X = AX+F 
X® = AXU-D4F 


it follows that 
X-X® = A(X-X-V), 
Hence 
|X— XH] = |A(X-XE-V|] < || A | |X-—XE-V]p. 


The vector norms I, II, HII, introduced by us in § 5, and the matrix 
norms subordinate to them, give the following easily tested sufficient 
tokens of convergence of the iterative process and estimates of the 
rate of its convergence. 
n 
I. If > |a;;|<y<1, the process of iteration converges, and 
Jol 
(6) Ij— af] < pmax |x;—2/-, 
3 


where X=(x,,...,%,) and X® = (x), 2, x), 


Ce y |a,;|<v<1, the process of iteration converges, and 
iS 
a > Ix; af? |< » > jx; —x-]. 
is] 2) 


Ill. If 5 a?,< P<, the process of iteration converges, and 
isl 
kal 


@ [Swamy s vel $ wate. 


Indeed, the largest proper number A, of the matrix A’A does not 
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exceed the sum of all the proper numbers of the matrix A’, for they 
are all non-negative. 
But 
Aytdet ++ $4, = te A’d = 3 (a)? = P 
rel 
Therefore |Al]11, =A, < VP <1, and consequently 





WX—XO la = J 2 (x; —x{")? < VP 5 (x, -x-))?. 


We shall indicate still another way of constructing sufficient 
tokens of the convergence of the iterative process. 
In the equation 


(9) X = AX+F 


with matrix 


(10) Behe odo ens ea 
Gynt 8g. +++ Any 


let us introduce new unknowns x; =9;2z;, where p; are some positive 
numbers. 
System (1) will then be transformed into the system 


(11) bits = 3, aubj2+h, 
or 
(12) a= 3 ay Ba 4ofu 


It is obvious that the components of the successive approximations 
X(#) for system (9) and Z“) for system (12) will also be connected by 
the relations x‘ =;z“, provided these relations hold for the initial 
approximations X( and Z‘. The iterative processes for systems 
(9) and (12) therefore converge or diverge simultaneously, and 
consequently any sufficient condition for convergence of the 
iterative process for system (12) will also be a sufficient condition of 
convergence for system (9). 
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Thus if one can find positive numbers p),..., p, such that one 
of the conditions 


1) > la,,| 2 <1 fori=l,...,2 
ist bj 
(13) 2) S lal SE < 1 for j=l,...,” 


n a2 p? 
3 <1! 
) on f? ; 
is satisfied, the process of iteration for system (9) will converge. 
Observation. In the practical computation of the iterations we 
may proceed by two methods: 


1) Put X=0. Then 
X® = (I+ At +++ +Al-')F = F+AF+ --- 4+Al-iF. 


For the computation of X) we calculate successively the vectors F, 

AF, ..., A‘-\F and find their sum. This is convenient, because of 

the uniformity of the process of computation, and also because each 

succeeding term is only a correction to the sum of the preceding 

ones. The shortcoming of this method is the possible accumulation 

of rounding errors with the increase in the number of summands. 
2) The computation is conducted directly by the formulas 


XW = AXU-D4F, 


Here each approximation is treated as a new initial one, and 
there is therefore no need in the first steps of the process to carry 
out the computations to any great accuracy; the errors that arise 
even themselves out later on. 

As an example let us find the solution of the system 


0.78x, —0.02x2 —0.12x3—0.14x, = 0.76 
—0.02x, +-0.86x, —0.04x3+0.06x, = 0.08 
—0.12x; —0.04x9 +0.72x3—0.08x, = 1.12 
—0.14x, +0.06x, —0.08x3 +0.74x, = 0.68. 


(14) 
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Solving this system by the single-division scheme, we find 


x, = 1.534965 
xy = 0.122010 
xq = 1.975156 
x, = 1.412955. 


For the application of the iterative process we bring the system 
into the form X=AX+F. It is possible to do this thus, for instance: 


X) = 0.22x, + 0.02%, +0.12x3+0.14x, + 0.76 
Xq = 0.02x,+0.14x_ +0.04x3 — 0.06x, + 0.08 
Xg = 0.12%) +0.04x5 +0.28x3 + 0.08x, + 1.12 
x, = 0.14x, —0.06x2 + 0.08x3 + 0.26x, + 0.68. 


(15) 


It is readily apparent that the sufficient conditions for convergence 
of the process of iteration are fulfilled. 

Let us compute the successive approximations by three methods, 
for a comparison of the course of the iterative process in the different 
variants, viz.: 


1) The computation of the successive approximations is effected 
k 
by the formula XY = 5 A/F (see Table XIII). 
150 


2) The computation of the successive approximations is effected 
by the formula X® = AX(¢k-) +F with X© =F (see Table XIV). 

3) Recompute X®=AX&-0+F with X(=(1,0,0,0) (see 
Table XV). 


Table XIII will be explained. The first part of the table contains 
the components of the successively computed vectors A/F. The last 


column is a check; in it are tabulated the numbers S b; xt », where 
b= Po a;; (these numbers must be calculated beforehand and 
elie in the same sheet as that on which the elements of the 


n n 
matrix A are tabulated). But it is obvious that > dx = > x(*), 
j=l jot 
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Tasce XIII. Iterative Method : Computation 


k 
by the Formula X= > AIF 
t=0 


1) AIF, =0, 1,..., 14 
F 0.76 0.08 1.12 0.68 


0.22 0.02 0.14 


0.02 0.14 0.04 0.06 


0.12 0.04 0.28 0.08 
0.14 - 0.06 0.08 0.26 


0.50 0.14 0.52 0.42 


0.76 0.08 1,12 0.68 

0.3984 0.0304 0.4624 0.3680 
0.1952 6400 | 0.0086 4000 | 0.2079 3600 | 0.1866 2400 
0.0942 1056 | 0.0022 3488 | 0.0969 2928 | 0.0919 7568 
0.0452 7913 | 0.0005 5572 | 0.0458 9292 | 0.0447 2340 
0.0217 4095 | 0.0001 3570 | 0.0218 8361 | 0.0216 0525 


0.0104 3649 | 0.0000 3285 | 0.0104 7017 | 0.0104 0364 
0.0050 0961 | 0.0000 0792 | 0.0050 1763 | 0.0050 0170 


0.0024 0463 | 0.0000 0190 | 0.0024 0654 | 0.0024 0272 
0.0011 5422 | 0.0000 0046 | 0.0011 5468 | 0.0011 5376 


0.0005 5403 | 0.0000 0011 | 0.0005 5414 | 0.0005 5392 
0.0002 6593 | 0.0000 0003 | 0.0002 6596 | 0.0002 6591 
0.0001 2765 | 0.0000 0001 | 0.0001 2765 | 0.0001 2764 
0.0000 6127 0.0000 6127 | 0.0000 6127 
0.0000 2941 0.0000 2941 | 0.0000 2941 


k 
2) xwW = = AMF, =k = 12, 13, 14. 


XQ2) | 1.5348 4720 | 0.1220 0958 | 1.9750 3858 | 1.4128 3762 
XM3) | 1.5349 0847 | 0.1220 0958 | 1.9750 9985 | 1.4128 9889 


XQ4) | 1.5349 3788 | 0.1220 0958 | 1.9751 2926 | 1.4129 2830 





1.2592 

0.5984 6400 
0.2853 5040 
0.1364 5117 
0.0653 6551 


(5) 
0.0313 4316 


0.0150 3686 
(79) 
0.0072 1580 


0.0034 6312 


(20) 
0.0016 6219 
0.0007 9783 
0.0003 8295 
0.000) 838) 
0.0000 8823 
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so that the elements of the check column are also equal to the sum 
of the rest of the elements lying in the same row. The second part 
of the table gives the approximate solution of the system, which we 
obtain by summing up the corresponding components of the com- 
puted vectors. 

From a comparison of Tables XIII, XIV and XV with the result 
obtained by the single-division scheme, we see that the convergence 


Taste XIV. Iterative Method: Computation by 
the Formula X® =AX¢-4F; X® =F. 


0.22 0.14 
0.02 — 0.06 
0.12 . 0.08 


0.14 0.26 
“050 | 014 052 | 0.42 
rome ||| 008) | «1.2 068 
ryise¢ | O.llos | 1.5824 =| «1.0480 

1.3537 0.1190 1.7903 1.2346 

1.4479 0.1213 1.8873 1.3266 

1.4932 0.1218 1.9332 1.3713 


1.5149 0.1220 1.9551 1.3929 

1.5253 0.1220 1.9655 1.4033 

1.5303 0.1220 1.9705 1.4083 

1.53273 0.12201 1.97292 1.41072 5.03838 
1.53389 0.12201 1.97408 1.41188 5.041 & 
1.53445 0.12201 1.97464 1.41244 5.04354 
1.53472 0.12201 1.97491 1.41271 5.04454 
1.53485 0.12201 1.97504 1.41284 5.04493 
1.534910 0.122010 1.975101 1.412900 5.044920. 


15349385 | 0.1220096 1.9751299 1.4129289 | 5.0450069 
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of the process in all three variants is nearly identical: the fourteenth 
approximation gives, in the example in hand, a valid result with 
accuracy to a unit of the fourth decimal place. 

The convergence of the iterative process can be much improved 
by means of a simple device suggested by L. A. Liusternik [1]. 


Taste XV. Iterative Method: Computation by 
the Formula X'®) = AX¢-N+F; XO =(1, 0, 0, 0) 


L.2+ 

1.6544 
1.8266 
1.9049 


1.9416 

1.9591 1.3970 

1.9675 1.4053 5.0221 
1.53129 0.12201 1.97148 1.40928 5.03406 
1.53320 0.12201 1.97339 L4ll9 5.03979 
1.53412 0.12201 1.97431 1.41211 5.04259 
1.53456 0.12201 1.97475 1.41255 cies 
1.534770 0.122010 1.974962 1.412761 a) 
1.534872 0.122010 1.975063 1.412862 3.044806 


(3) 
1.534920 0.122010 1.975112 1.412911 5.044952 
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In view of the fact that this device is connected with the approxi- 
mate computation of the first proper number of the matrix, we post- 
pone consideration of it until § 35, in Chapter IIT. 


§ 18. THE PREPARATORY CONVERSION OF A 
SYSTEM OF LINEAR EQUATIONS INTO 
FORM SUITABLE FOR THE METHOD OF 
ITERATION 


In the preceding section we have established a number of 
sufficient conditions for the convergence of the process of iteration, 
as applied to systems of equations given in the form 


(1) X = AX4+F 
or, what is the same thing, 
(1-A)X = F. 


All those criteria required that the matrix -l be small in one sense 
or another, i.c., that in the matrix (/—A) the diagonal elements be 
sufficiently preponderant and be close to unity. 

Let us consider some devices for transforming a given system 
BX =G into form (1) with a suitable matrix A. 

A system 


(2) BX =G 


is especially easily transformed into the necessary form in case the 
diagonal elements of the matrix B considerably preponderate over 
the remaining elements. 

Let us rewrite system (2) in extended form: 


yx t+ byoxet +++ +bigk, = 81 
baiX + borg t +--+ +baq%, = Bo 
(3) arn = 8 


a i i a Y 


by*1 + byoxe as + baaXn = Sn 
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Divide each equation of system (3) by the diagonal element. We 
obtain the system 


Sra aides ging. Bt 

bin bu 11 

boy bo, 9 

—r +x + . + pe J Xn = oS 

(4) bog 1°? bee bus 


oe © © © © © © © © © © © © © 


bat bay Sn 
— x. oe a a 
ban . bun 


This system, in matrix form, is given notationally by 


(5) X = AX+F, 
where 

o | _ bin fr 
by bn 11 
_ $21 O- es% — ben 22 
(6) A _ bse bos ? F = bao 
bat _ ng &n 
Ban an : ban 


For application of the process of iteration there is no need to 
actually make the transformation of (3) into system (5). The suc- 
cessive approximations may be computed by the formulas 


Qj. “He ished fete eee 


bX) = gy byt D— os d,s. 
The modification of the iterative process just described bears the 
appellation method of simple iteration. 


The sufficient conditions of convergence of the iterative process 
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that were deduced in § 17, on being applied to system (5), give the 
following sufficient conditions for the method of simple iteration: 


b 








L > uy <i! 
G21 | 

(8) II. 3 | 5 <1 
i= | 5, 


Wn. + (32)" ZA. 


Ese Vii 


Here the prime sign on the sum signifies that the value i=) is 
omitted in the summation. 
If one makes use of the generalized criteria (13) § 17, having put 


p= ier we shall then obtain the following additional sufficient 


tokens of the convergence of the simple iteration: 





Seba ct 
sah Os 

(8’) I’. || <1 
f=1 


The transformation of system (2) into system (5) just mentioned is 
obviously equivalent to the multiplication of system (2) on the left 
by the matrix 


en 
bi 
= 6 


boo 
(9) D= 
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We thereby pass to the system DBX=DG, which we write in the 
form X= (I—DB)X+ DG. 
Assume now that a system 


BX=G 


has been given, in which a predominance of the principal diagonal 
does not obtain. For application of the iterative process this system 
must be transformed into an equivalent system of the form 


(I-A)X = F, 


with the matrix J—A having a predominant principal diagonal. 
For this it is enough to premultiply both sides of the equation BX =G 
by an auxiliary matrix D sufficiently close to the matrix B-!, 
Indeed, after such a multiplication we will obtain the system 


DBX = DG, 


in which the matrix DB is close to BB=/. Let DB=/—A, 
DG=F. Since the matrix A will be small here, this transformation 
will bring the given system into the required form. 

The selection of the auxiliary matrix D may be accomplished by 
various means, for instance by a rough inversion of the matrix B by 
the Gauss method. In case the predominance of the diagonal 
elements in the original system BX =G does exist, but is insufficient, 
it is usually expedient to take as D the matrix inverse to the matrix 


by bie 
boy boo 
bsg bag 


bas bag 
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The inversion of such a matrix presents no difficulty, for it reduces 
to the inversion of matrices of the second order, viz.: 


boo _ biz 
4 A 
wa’ bn 
D = A, A, ? 
; . bi, Oye 
where 4, is the determinant | _, , : 
21 %99 





We note that the necessary and sufficient condition for the con- 
vergence of the iterations in application to the preparcd system 


DBX = DG 


consists in all proper numbers of the matrix J— DB being less than 
unity. 


§ 19. SEIDEL’S METHOD 


Let a system of linear equations be given in the form 
(1) X = AX+F, 


where A is the given matrix, F the given vector and X the sought 
vector with components (x), ..., *,)- 

The iterative method of Seidel is reminiscent of the ordinary 
iterative process, with this difference, that in computing the &th 
approximation to the component x;, one takes into consideration 
the kth approximations, already computed, to the components 
¥1,++.+,%;-1- Explicitly, the computation of the successive approxi- 
mations is performed by the formulas 


i-1 _ 
(2) x) = Pa ax) + 2 a,xt—) +); 


n 
(instead of by xf) = > a,x!*-") with the method of iteration). 
i511 
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In a number of cases it turns out that the Seidel process converges 
faster than the ordinary process of iteration. 
Such is the case, in particular, on condition that 


|All; = ee 2 la; < B< I. 


Indeed, as we have seen, in this case there holds for the method of 
iteration the estimate 


(3) |X-XO|] < pllX—XE-], 
where as the norm of vectors the first norm is taken, i.e., max |x;]. 


We shall show that under this condition the Seidel process con- 
verges, and that a somewhat better estimate obtains for it. 
Indeed, if x,,..., x, is the solution of system (1), then 


(4) x= Ds a;x;+f;, t=1,...2. 
Subtracting equation (2) from (4), we obtain 
x,—x) = = a,,(x;,— x!) + 3, a4l5-9-) 

whence it follows that 
(5) bal < Slagle alt She bal 
Let us employ the symbols 

2 la;| = Bi, > la;| = Ye 
Then from inequality (5) it follows that 


|x; — {| < Ai|X-X®|]+ y|]X-X4)]. 


Taking for i that value ig for which |x; —x(| attains its maximum, 
we obtain 


(6) |X—X®)| < kr |X— X8-D |], 


for |X—X || =max |x, —24| tier 
1 
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Let us use the designation 


(7) max eat = pb. 





Then 
(8) |X—X®]] < p'|X—-X4E-V]], 
We shall establish that 
) Ws 
Indeed, 
p> la,,| = Bi+yi 

and 

Biri T oe — 6 (1-8: - yi) > 0, 


‘ 1 i 
for Bi +y;<n<1. Hence 


fe = max (8; +y;) 2 max a =p’. 





n 
Here the equality sign is possible only if max > |a,,| is attained for 
i jel 
t= 1, and the reduction of the estimate in comparison with estimate 


(3) will be best if the equations be arranged in increasing order of 
the > |a,;|, adopting as the first that equation in which this sum is 
j=) 


least. 

The Seidel method nevertheless does not always prove to have 
the advantage over the ordinary iterative process. 

Sometimes the Seidel process converges more slowly than the 
process of iteration. It is even possible that the Seidel method may 
diverge while the method of iteration converges. The regions of 
convergence of these two processes are different, overlapping only 
partially. 

We shall now establish the relation between the Seidel method 
and the method of iteration, which will make it possible to indicate 
the necessary and sufficient condition of convergence of the Seidel 
process. 
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Let us represent the equation 


X = AX+F 
in the form 
(10) X = (B+AC)X+F, 
where 
(11) 
0 0 0 @11 @\2 a1, 
pu| ™ 9 0 Ge 0 age Fey 
ani ano eae a, a-1 0 0 0 eve ann 


With these symbols one may represent the formulas 
in matrix form as 
(12) X® = BXW4CXU-04F, 
Hence it follows that 

X® = (1—B)-'1CX&-) + (I—B)-1F, 


Thus the Seidel method for system (10) is equivalent to the applica- 
tion of the method of iteration to the system 


(13) X = (I—B)-'CX 4+ (I—B)-'F, 
which is equivalent to an initial system 
X = (B+C)X+F 


and may be obtained from it by premultiplication by the non- 
singular matrix ({—B)-}. 

From such a representation of the Seidel method it follows that 
for its convergence it is necessary and sufficient that all proper 
numbers of the matrix M = (J—B)-C be less than unity in modulus. 
These proper numbers are the roots of the equation |M—Ai|=0. 
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Multiplying both sides of the last equation by |J—B| and utilizing 
the theorem concerning the product of the determinants of two 
matrices, we transform the equation into the form 


|C—(I-B)A| = 0, 


or, in extended form, 


a1—-A aye WN» 
(14) 493A aa9—A ore Aon -0. 
aA a,0A Oag—A 


4 
Thus for convergence of the Seidel method, it is necessary and suff- 
cient that all roots of the equation (14) be of modulus léss than 
unity. 

We shall now offer examples exhibiting the difference between the 
regions of convergence of Seidel’s method and the ordinary method 
of iteration. 

Example 1. Let 


The proper numbers of the matrix A will then be determined from 
the equation (0.1—A)(5-—4)+5=0, and therefore |A;|>1. The 
ordinary process of iteration diverges. 

Let us form the matrix 


woee(} 38) 


The proper numbers of this matrix are determined from the equation 
A2—0.144+0.5 = 0. 


Obviously |A;|<1. The Seidel process converges. 
Example 2. Let 
A= ( —5 ): 


1 -2.3 


136 Systems of Linear Equations 


The proper numbers of the matrix A are determined from the 
equation 


—(2.3—2)(2.34+4)+5 = A2-0.29 = 0; [A,|<1. 


The ordinary process of iteration converges. 

It is readily verified that in this case the Seidel method diverges, 
for indeed, (J—B)-1C =( sie ae ) , and the proper numbers of 
this matrix are of modulus greater than unity. 

If the system of equations is given in the form 


(15) MX = G, 


then in order to apply the Seidel method this system must be sub- 
jected to preparation to make it over into the form 


X = AX+F. 
The method most used leads to a process parallel to the process 


used for simple iteration, viz.: the system MX=G is tabulated in the 
form 


i-1 a 
mx, = — Dd mixj;—- 2D miyxj+e; 
j=l joitl 
or, what is the same thing, in the form 


oe no om: . 
a x, Ti x + Sh 
jal My; sortl Mj; mi; 


and the successive approximations are determined by the formulas 


i-l m.. 2 om.; &; 
16 = — 5 Tio y By ey Si 
ve) : A m; 7 inti m, / mij 


From the necessary and sufficient conditions of convergence in the 
general case considered above, those for the convergence of this 
modification of the process may be readily obtained, to wit: for the 
convergence of the process it is necessary and sufficient that all 
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proper numbers of the matrix Q-'R be less than unity in modulus. 


Here 


(17) 
my) 0 0 0 Mio mi, 
as Mz, Mag . 0 R= 0 Oo .. Me, 
My, My sae M,, 0 0 0 


Indeed, the chosen preparation by which the system MX=G is 
transformed into X = AX + Fis equivalent toa premultiplication of the 
system by a diagonal matrix P-!, where P is the matrix composed 
of the elements of the principal diagonal of matrix Af. The matrix 
P-1Q thereby plays the role of the matrix (J—B), and the matrix 
—P-'R the role of the matrix C, and consequently (/—B)-1C 
= —Q-!R; the proper numbers of the matrices —Q-'R and Q-!R 
differ only in sign. 

Furthermore, the characteristic polynomial |— Q-!—A/| of the 
matrix —Q-)2R, after multiplication by |—@Q|, acquires the form 
|R+A4Q|. Consequently for the convergence of the specified modi- 
fication of the Seidel process it is necessary and sufficient that all 
roots of the equation 


mA Myo see My, 


MayA mad... Mag 


(18) 


co ee 


mA mA ... mad 


be of modulus less than one. 

A great number of sufficient criteria of the convergence of this 
modification of the Seidel method is given in a list by P. A. Nekrasov 
and R. Mehmkc.! 

In particular, criteria I, IH, I‘, I’ of § 18 are sufficient. 

In case the matrix M of the coefficients of the system MX = G is 
symmetric, there exists still another important sufficient condition 


1 R. Mehmke and P. A. Nekrasov (1), P. A. Nekrasov [I]. 
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of convergence of the modification of Seidel’s process under con- 
sideration, viz.: 

Uf the quadratic form with matrix M is positive-definite, the Setdel method 
Jor the system MX=G converges. This condition, on the assumption that 
the diagonal elements of the matrix M are positive, proves to be necessary as 
well. 

For the proof we shall put 


(19) M = P+Q+Q’, 


where P is the diagonal matrix composed of the diagonal elements 
of the matrix M; Q is a triangular matrix formed by the elements of 
the matrix M that are located below the principal diagonal; and 
Q’ is the matrix of the elements of M located above the principal 
diagonal. On the strength of the symmetry of M, Q’ is the 
transpose of the matrix Q. 

As we have seen above, the necessary and sufficient condition of 
convergence of the Seidel method is the requirement that all proper 
numbers of the matrix T= (P+ Q)-1Q’ be of modulus less than one. 

Let y; and y; be any two proper numbers of the matrix T (they 
may, generally speaking, be complex), and let Z; and Z; be the 
proper vectors belonging to them. Then 


(P+Q)-1Q'Z; = 4;Z,, 
whence 


(20) Q'Z; = wjPZ;+4;QZ;. 


Analogously 
Q'Z; = wjPZ;+4jQZ;. 

We shall establish a relation of dependence between the scalar 
products (MZ;, Z;) and (PZ;, Z;), which will be important to all 
the subsequent discussion. 

To this end let us express (Q'Z;, Z;) and (QZ,, Z;) beforehand 
in terms of (PZ;, Z;). 

We conclude from relation (20) that 
(21) (Q'Z;, Z;) = u(PZ;, Z;)+4(QZ;, Z;). 

1 E, Reich [I]. 
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Analogously 
(22) (Q'Z;, Z;) = w(PZ;, Z;) +4 (QZ;, Z,). 
But, on the basis of the properties of the scalar product: 
(QZ; Z;) = (Z, QZ;) = (QZ;, Z)) 
(23) (QZ;, Z;) = (Z;, Q'Z,) = (Q'Z;, Z)) Fh 
(PZ, Z;) = (Zp PZ;) = (PZ, Z). 
Using these relations, let us write equation (22) in the form 
(QZ; Z) = wlPZ, Z) +u(PZ, Z) 
and, passing to the complex-conjugates, 
(24) (Q2Z;, 2;) = A(PZ,, Z;)+4,;(Q'Z;, Z;). 
From (21) and (24) we find 
(1 +42;) 
Z, Z)) = AL+H) (pz, z, 
(92, 2) = GOH (pz, Z) 
and 


a 


(Q°Z;, Z,) = HU +h) (pz, Z,), 


We have, moreover, 


(MZ;, Z;) = (PZ;, Z;)+(QZ,, Z;) +(Q'Z;, Z;) 


(25) = 142i te) 4 ull tA) (pz, z,) 
I~pe; | —pif; 


= ae (PZ;, Z;). 


From the last relation the sufficiency of the formulated condition 
for the convergencc of Seidel’s method follows directly. 
Indecd, putting i=/, we obtain 


— (+H) +4) (PZ, Zi), 


i> i) 


l-p,fi; 
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But 

a; = |ajl?; (l+ei)(1+a,) = [1 +4,)% 

Thus 
1+4;|2(PZ;, Z,) 
—lnj2 = | i i> 

ul? = (MZ,, Zz: i) > 0, 
for in accordance with the property of a positive-definite quadratic 
form, (MZ,, Z;)>0 (see §3, Paragraph 8), just as (PZ;, Z;) >0. 
The last inequality follows from the fact that the diagonal terms of 
a positive-definite quadratic form themselves constitute a positive- 
definite quadratic form. 

Proof of the necessity of the condition is somewhat more complex 
and is based on the following lemma from the theory of Hermitian 
forms. 

Lemma. If 


a 
B(x1, Xo)... Xq) = ; 2, C4jxi; 
LT Dd 


is a positive-definite Hermitian form, «1, @,...,¢@, are arbitrary 
complex numbers and 4, Bays ess Hn complex numbers less than of 
unit modulus, then 


(26) W(x, X25 066 hy) = > Ci os x5; 
3 


igal 


is also a positive-definite Hermitian form. 
Proof. Indeed, on the strength of the fact that all |y;| <1, 


tS =e Ha, 


whence 


o aA 
W=)D D> C; iO te ii aed; 
=0 jel 


= 2 Peatity Co [5X 9, se eey a, #3X,) >0, 


for all terms of the series in question are positive. 
The lemma is proved. 
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Let us return to the proof of the necessity of the formulated condi- 
tion of convergence of the Seidel method, on the assumption of the 
positiveness of the diagonal elements of the matrix M. 

Let the Seidel process converge. Then all proper numbers yz; of 
the matrix (P+ @)-!Q’, are of modulus less than unity. Let us 
assume, in addition, that they are all distinct, pair by pair. On 
this assumption the proper vectors Z;, Zo,..., Z, are linearly 
independent and may be taken as a basis of the space. 

Let X be any vector of the space and let x,...,.%, be its co- 
ordinates on the basis Z), Z2,..., Z,, that is 


xX = xZ,+ eee +x,Z,- 


We now introduce into the discussion the quadratic form 
W=(MX, X). In terms of the chosen coordinates it is expressible 
as an Hermitian form: 

Q7) w= S xa(Mz,z) = 5 CtwU FH), 
fei agai lpi; 
(on the basis of equation (25), where ¢;;= (PZ;, Z;)). 
The form 


is positive-definite, on the strength of the assumption of the positive- 
ness of all elements of the diagonal matrix P. 

Accordingly, on the strength of the lemma, the form ¥ is also 
positive-definite, which is what was required to be proved. 

The assumption that was made regarding the absence of multiple 
proper numbers of the matrix T is not essential, for its satisfaction 
can be accomplished by an arbitrarily small deformation of the 
elements of the matrix M. Therefore from considerations of con- 
tinuity , the positiveness of the form (MX, X) turns out to be a 
necessary condition of the convergence of the Seidel method (on 
the assumption that the diagonal elements of the matrix M are 
positive, of course) even given that multiple proper numbers are 
present. 

We remark in conclusion that the method known as the “ relaxation 
method”’ for the solution of linear systems is nothing more than an 
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alteration of the computational scheme of the Seidel method. On 
this subject we refer the reader to an article by M. V. Nikolayeva.! 

As an example let us find the solution of system (14) in § 17, 
having brought it into the form 


X, = 0.22x,+0.02x0 +0.12x3 +0.14x, + 0.76 
Xq = 0.02x,; +0.14x_ +0.04x3 — 0.06x, +0.08 
Xg = 0.12x,+0.04x2 +0.28x3 + 0.08x, + 1.12 
x, = 0.14x, —0.06x2+0.08x3 + 0.26x, + 0.68. 


The sufficient conditions for the convergence of the Seidel process 
are obviously fulfilled. 


TasBLe XVI. Computation of the Solution of the 
System by Seidel’s Method 


F=X0) 


1.53470 0.12200 1.97494 1.41281 
1.53486 0.12201 1.97507 1.41290 
1.53492 0.12201 1.97512 1.41293 
1.534947 0.122009 1.975142 1.412945 
1.5349579 0.1220094 1.9751507 1.4129513 
1.5349622 0.1220095 1.9751541 1.4129538 





1M. V. Nikolayeva [1]. 


° 
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As the initial approximation we will take the vector of constant 
terms. The successive approximations are set down in Table XVI. 

Comparing the successive approximations that we have found 
with the solution of the system as found by the single-division 
method (see § 17), we see that the fourteenth approximation gives 
the solution to an accuracy of three units in the sixth place. The 
Seidel process, in the example before us, converges more rapidly 
than the ordinary process of iteration (see Table XIII, XIV and XV 


$17). 


§ 20. COMPARISON OF THE METHODS 


In concluding this chapter we will make a comparison of the 
methods described, as regards their application to the problem of 
finding the solution of a non-homogencous linear system. 

The generally accepted criterion of the ‘‘advantageousness”’ of a 
given computational scheme is the number of necessary computa- 
tional operations, that is, the number of necessary multiplications 
and divisions, and also additions and subtractions. Fundamental 
to this is the number of multiplications and divisions, and therefore 
in characterizing a method from this point of view we have limited 
ourselves to reckoning this number only. These criteria may not, 
however, be the only ones. Simplicity and uniformity of the 
operations to be performed in conformity with a given scheme may 
compel the computor to prefer it to a scheme with a considerably 
lesser number of operations but of more complicated “pattern”’. 
The latter circumstance is particularly important in case the com- 
putor has a digital calculator at his disposal. 

Furthermore, onc must take into consideration the possibility of 
a compact recording of the intermediate results, i.c., the possibility 
of carrying through the computations by accumulation formulas. 
Compact schemes, although they require higher qualifications on 
the part of the computor, frequently much simplify the entire process 
of computation. An important factor influencing the choice of a 
computational scheme is the absence of a loss of significant figures in 
the process of computation by a given scheme. Finally, the decisive 
factor is the reliability of the results obtained. 
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Let-us analyse the methods we have described from the point of 
view of the factors indicated. 

As was remarked earlier, the iterative methods offer greater 
simplicity in respect of the structure of their computation schemes. 
The separate operations are also carried out by accumulative means, 
and the computation can be carried through by a self-correcting 
process. Here it is easy to determine the number of operations 
necessary to obtain one iteration; to determine, however, the num- 
ber & of necessary iterations is ordinarily possible only in the course 
of the work; it may prove to be very large. 

Therefore in case the form of the system’s coefficient matrix per- 
mits one to have confidence that the convergence of the iterative 
process will be sufficiently rapid (say k<n, where n is the order of 
the matrix of the system), this method may well be preferred to all 
others. Together with it one should without fail, of course, utilize 
the device that we analyse in § 35 for improving the convergence of 
the process. 

Systems, however, exist for which the generally adopted iterative 
methods either converge too slowly or even diverge. In this case 
only the exact methods make possible the computation of the 
solution. 

Modifications of the most powerful of the exact methods—the 
Gauss method—are the compact schemes, those of the type of the 
single-division scheme, and the elimination scheme. The compact 
schemes of the Gauss method, and in particular the square-root 
method for systems with symmetric matrix, can be recommended 
in case highly qualified computors are available. The single- 
division scheme may sometimes be successfully rivalled by the 
elimination scheme, in which the whole process of computations is 
conducted uniformly, thanks to the absence of a return course. 

The number of operations by the different variants of the Gauss 
method is approximately the same; only the number of results to be 
recorded are different. As has already been remarked in describing 
the Gauss method, its basic defect is the possible disappearance of 
significant figures in case either the determinant of the system itself 
or that of one of the intermediate minors is small. 

In such cases the escalator method is reliable (we have described 
it only for symmetric systems). 
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Finally, the determination of the solution by means of the inverse 
matrix, theoretically the simplest, is in practice sometimes also the 
most efficient method. 

This will be the case, for example, when the solution must be 
determined with a high degree of accuracy. Indeed, as we saw in 
§ 13, the elements of the inverse matrix can be determined as 
exactly as one wishes, as scon as onc has found a sufficiently exact 
approximation, from which an iterative process giving these elements 
converges extremely rapidly. For computing the first approxima- 
tion to the inverse matrix, along with the method of Gauss, the 
method of bordering may also be employed with success. 


CHAPTER III 


THE PROPER NUMBERS AND PROPER 
VECTORS OF A MATRIX 


This chapter is devoted to the problems of finding the proper 
numbers and the proper vectors of a matrix. We recall that what 
are spoken of as the proper numbers of a matrix A are the zeros of 
its characteristic polynomial, i.e., the roots of the equation 


a4,—A ayo eee a), 

aa, @o9—A ao 

|4—Al| = . 
a) G9 we Ogg A 


(=I)sfde— part +++ -p,] = 0. 


As has already been remarked in § 1, the coefficients p; are, but 
for sign, the sums of all principal :th order minors of the determinant 
of the matrix A. The direct computation of the coefficients 9; is 
extremely awkward and requires a huge number of operations. 

The determination of the components of a proper vector requires 
the solution of a system of n homogeneous equations in 2 unknowns; 
in order to compute all the vectors of a matrix, one must solve, 
generally speaking, n systems of the form 


(A—A,1)X, = 0, 


where X; = (x);, oj) - » - » Xqi) is the ith proper vector of the matrix A. 
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It is thus perfectly natural that special computational artifices 
that simplify the numerical solution of both the problems before us 
should have made their appearance. As in the preceding chapter, 
we shall distinguish two groups of methods: exact and iterative. 
The exact methods, when applied to the first problem, will give 
more or less convenient schemes for determining the coefficients p;. 
The proper numbers will then be obtained as the solutions of an 
algebraic equation of the nth degree. We note that all of the 
methods proposed below, with the exception of that of Leverrier 
(1840), have appeared in the thirties of our century or later. The 
majority of the methods proposed are adapted to the solution of the 
first problem. We shall show, however, that by utilizing the inter- 
mediate results of the computations, the proper vectors of the matrix 
belonging to the corresponding proper numbers can also be com- 
puted after the latter (avoiding the solution of the systems indicated 
above). Only one method—namely that known as the escalator 
method—solves both problems simultaneously, but it makes neces- 
sary the computation, in the course of the process, of the complete 
spectrum of the matrix, and all the proper vectors not only of the 
given matrix but of its transpose as well.! 

The iterative methods make possible the direct determination of 
the proper numbers of the matrix, without resorting to the char- 
acteristic polynomial. In using them, only the first proper numbers 
are as a rule determined with sufficient accuracy. The course of 
the iterative process depends essentially on the structure of the 
Jordan canonical form that is connected with the given matrix and 
on whether the proper numbers of the matrix are real or imaginary. 


1 We note that the determination of the roots of the characteristic equation may 
be efficiently carried out by Newton’s method, viz.: if 


9(a) = IP—pydP-1— «+ —py = 0 
@(Ao) 
# (Ao) 
approximation to a root, and A, gives a significantly better approximation than Ap. 
The values of p(49) are conveniently calculated by Horner’s method: ~(A) = gn, 
where the numbers q (£=0,..., 2) are computed by the recurrence formulas 


% = M-Ao— his 9-1 = 0. 


is the characteristic equation, then A,=1g— . where dg is some initial 


Analogously 
P (Ao) = ta-ay Where = -wAo tg f-1 = 0. 
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The convergence of the iterative process is determined by the mag- 
nitude of the ratio of the moduli of different, neighboring proper 
numbers, and may be very slow. We shall exhibit several artifices 
making it possible to much improve the convergence of the process. 
A proper vector is computed simultaneously with the proper number 
to which it belongs. As in the solution of linear systems, the chief 
merit of the iterative processes consists in the simplicity and uni- - 
formity of the operations to be performed. 

In all the following sections we shall assume the coefficients of 
the matrix to be real. 


§21. THE METHOD OF A. N. KRYLOV 


A. N. Krylov’s work [1] was the first in a great cycle of works 
devoted to bringing the secular equation into polynomial form. 
The idea of A. N. Krylov consists in making a preliminary trans- 
formation of the equation 


ay,—A aye eee ay, 
D e—A eee 4 
(1) D(a) = 421 422 Min | 0 
ani G2 +++ Ang—A 


by, -A Dio bin 
bo, — 22 29 

@) D(a) = [PB bee vee eet 
by, — A" bye Onn 


which is, generally speaking, equivalent to it, and whose expansion 
in powers of 4 is obviously accomplished considerably more simply by 
expanding this determinant by minors of the first column. 

In order to effect the transformation indicated, A. N. Krylov 
introduces into the discussion a differential equation, which is con- 
nected with the given matrix; he simultaneously poses the problem 
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of finding a purely algebraic transformation turning equation (1) 
into equation (2). 

Works of N. N. Luzin [1], [2], I. N. Khlodovsky [1], F. R. Gant- 
makher [1], and D. K. Faddeev [2] are devoted to explaining the 
algebraic essence of the transformation of A. N. Krylov’s. 

We shall present A. N. Krylov’s method in an algebraic inter- 
pretation. 

That the determinant 


@yi—A ayy wey 
I nl eT 
Gar ne Sie | 


equals zero is the necessary and sufficient condition that the system 
of homogeneous equations 


Axy = Qyy%y+Qyo%o+ +++ +415%, 


(4) Akg = doyX, +agotet +++ +GoqX, 


Ax, = 4X1 + Anoret 6° + FagaXy 


have a solution x), x9,..., x, different from zero. 

Let us transform system (4) in the following manner. We shall 
multiply the first equation by 4 and replace Ax,,..., dx, by their 
expressions (4) in terms of x1, ... 5 Xq- 


This gives 
(8) ts Sie a as 
where 
(6) by, = 2 4) yp: 


Moreover, Jet us multiply equation (5) by 4 and again replace 
Ax,, Axo,..., Ax, by their expressions in terms of x;,...,x,. We 
shall obtain 


AS8x, = b31x1 + bge%et ++ > tbagky: 
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Repeating this process (n—1) times, we pass from system (4) to 
the system 


Any = byyxy + byoxat +++ +bigk, 
Ax, = bo3%1 + beox0+ he (8 + ba,%_ 


(7) 


Arxy = bart1 + boxe + ahi + bankas 


whose coefficients 4;, will be determined by the recurrence formulas 


b. = 2 bi-1, Ost 


2, 
(8) , 


2 


t=2,...," 
k=l,...,2” 


by = ay. 

The determinant of system (7) will obviously have form (2). 
System (7) has a non-zero solution for all values of A satisfying the 
equation D(A)=0. Thus D,(A) reduces to zero for all 4 that are 


roots of the equation D(A) =0. 
We shall show that 





1 0 0 
D(a) _| 1 big wee a 
) Dy a Wes agit Coteus Se Bea 


b,-1, 1 b,-1, | ee b,-1, a 


that is, for N#0, D,(A) differs from the sought characteristic poly- 
nomial by a numerical factor only. 

Let all roots of D(A) be distinct. Since all roots of D(A) are roots 
of D, (A), D,(A) is divisible by D(A). Since, in addition, the degrees 
of D,(A) and D(A) are alike, the quotient must be a constant (does not 
depend on 4). Comparing the coefficients of A", we obtain 


D,(A) 


Day = 





In case D(A) has multiple roots, the equation 


(10) D(a) = ND(A) 
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is preserved, which follows, however, from considerations of con- 
tinuity. 

This equation may also be verified directly by a multiplication of 
the determinants figuring in it if relations (8) are used. 

It is evident from equation (10) that if N=0, D,(A) is identically 
equal to zero. In this case the indicated transformation yields 
nothing. ; 

However even for N=0, A. N. Krylov proposes a special method 
whose algebraic substance will be elucidated below. 

Let us now turn to the coefficients 6;, defining D,(4). We 
introduce into the discussion the vectors B, with components 
(b;,, bya)... , 8;,). The equations 


by eS Bek cay ee 
(8) aaa aa 
bags =),...,2, 
show that 
(11) B; = A‘B;_3, 


A’ being the transpose of the given matrix. 
From equation (11) it follows that 


B; = A'%-1B,, i=2,...,2. 
In its turn B, = A’Bo, where B= (1,0,...,0). Thus, finally, 
(12) B; = A''By, i=1,2,...,n. 

It is obvious that system (4) may be transformed by starting, for 
example, from the second equation of that system. In this case 4 
will be put into the second column of the determinant D,(4), and 
the coefficients 5,, will be determined by formulas (12), where 
Bo=(0, 1, 0,..., 0). 

The method of A. N. Krylov is quite naturally generalized if we 
introduce into the discussion instead of the vector By of special form, 


the arbitrary vector By=(bo1) boo, » - + » Bon) 
Let 


(13) U = box +booxet +++ + bon%ys 


where (1, X2,..-, *,) is the solution of system (4). 


fot O88 "> Dow u 
“> bn b. [* \ 2c 
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Then on repeating the preceding argument, we will obtain 
U = box, + boexet +++ +bon%, 
Au = bx, +bo%e+ +++ +O4,%, 
(14) A2u = bax) +booxat +++ +bo,X, 


Aru = b,1%1 + 5,0%9 + Tovey. +b uaXay 


where B; = (b;1y 5:2) « «+ 5 Bin) = A'Bo. 

Regarding the n+1 equations (14) as a system of homogencous 
linear equations in n+] unknowns u, x;,...,x,, we will have a 
non-zero solution possible in case, and only in case, the determinant 


el bon bon 
by we. 

(15) D,(a) = *) = 0, 
"An ba Dan 


Repeating the previous reasoning, we find that 


D,(4) = D(A)N, 


where this time 


bor boo bon 
b,-1, 1 b,-1, i wees b,-1, a 


Just as for the particular case considered above, the transforma- 
tion yields nothing if N=0. 

Let us assume at the outset, therefore, that N#0. On the basis 
of the equation D,(A) = ND(A), the coefficients p; of the character- 


(—1)2-1N;, 
N 


istic polynomial are to be determined as the ratios » where 


N, are the algebraic complements of the elements A*-i in the deter- 
minant D,(A). And the determination of the coefficients of the 


154 Proper Numbers and Proper Vectors of a Matrix 


characteristic polynomial by means of the ratios indicated indeed 
constitutes the essence of A. N. Krylov’s work. However, the 
investigation there conducted makes possible the determination of 
the sought-for coefficients while avoiding the computation of the 
minors, thus much reducing the number of operations necessary. 
Indeed, in view of the fact that the rows of determinant (16) are 
the components of the vectors By, B,,..., B,_3, the condition N40 
is equivalent to the linear independence of these vectors. Therefore 
for N#0, the vectors Bo, By,..., B,-1 form a basis of the space. 
Accordingly the vector B, is a linear combination of them: 


(17) 8, = nB yee eve +49,Bo. 


We shall show that the coefficients of this dependence are indeed the 
coefficients », of the characteristic polynomial, the latter being 
written in the form 


D(A) = (= 1)e[a—pydert— + - py. 


Indeed, taking from the last row of the determinant D,(A) a linear 
combination of the preceding rows with respective coefficients 
91s J2) - ++» Jn» WE Obtain, on the basis of equation (17), 


l bor bon 
Pula) < And b,-1,1 eccaie bat, 1 
da-t 
BP chicabtcreee Foe 4 
ase 
4 = (—1)#{As—g,An-1 — see —Q|N. 
Hence 


D(a) = Pl® = (—aypfar—guaent— «+ a), 


which is what was required to be proved. 

Equation (17) permits the finding of the coefficients ¢,=/;, 
J2=fo) +++» %,=P, as the solutions of a system of linear equations 
equivalent to this vector equation. 

Equation (17) connects the method of A. N. Krylov with the 
Cayley-Hamilton relation (applied to the matrix A’). 
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Indeed, from the relation 


A’n = pyAom-l yee tpl 
it follows that 
A'Bo = pi A Bot ++ +p,Bo, 
that is, that 


(17) B, = piBy-1t +++ +PnBo. 


In place of system (17), one can obviously employ for the deter- 
mination of the coefficients p; the system 


(17’) Cy = PiCy-1+ +++ +P,Co, 


where the vectors C, are determined by the equations C, = A'Co. 

For the determination of the coefficients p; by solving system (17) 
or (17'), one must carry out 3n2(2+ 1) multiplications and divisions. 
In its original form the method of A. N. Krylov required 
3(n4 + 4n3 4 2n2 —n—3) multiplications and divisions. 

In case N=0, the system equivalent to equation (17) docs not 
make possible the determination of the coefficients of the character- 
istic polynomial, since the determinant of this system is likewise 
equal to N. 

The algebraic essence of the device of A. N. Krylov’s alluded to 
above consists in this: it is possible to determine the coefficients of 
the polynomial of least degree, Y/(4), such that ¥/(A)Cg=0. Gener- 
ally speaking, this will be the minimum polynomial of the matrix, 
and its roots will coincide with all the roots of the characteristic 
polynomial, but will be of lower multiplicity. However, with an 
unfortunate choice of the vector Co, instead of the minimum poly- 
nomial some once of its divisors may be obtained, whereupon part 
of the roots of the equation |A—A/|=0 may be lost. As has been 
shown by N. N. Luzin and I. N. Khlodovsky,! with a special choice 
of the vector Cg one can obtain as the polynomial ¥/(A) any divisor 
of the minimum polynomial. 

We note that if the minimum polynomial of the matrix does not 
coincide with the characteristic polynomial, N=0 for any choice of 


YN. N. Luzin (1], (2]; 1. N. Khlodovsky [1]; F. R. Gantmakher {1}; D. K. 
Faddeev (2). 
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the vector Co, for Y(A)Cy=0, and since the degree of the poly- 
nomial ¥(A) is less than n, the vectors Co, ACo,..., A"-1Cy are 
linearly dependent. As regards an additional degeneration, this 
may be avoided by changing the initial vector Co. 

Thus the method of A. N. Krylov makes possible the determina- 
tion of the coefficients of the characteristic polynomial if N40, or 
of some one of its divisors—generally speaking the minimum 
polynomial—if N=0. 

In practice the circumstance that N=0 will reveal itself during 
the forward course of the solution of system (17) by the Gauss 
method. For in part of the equations all coefficients will be 
simultaneously eliminated, so that these equations will reduce to the 
identity 0=0. These equations (let them be n—m in number) 
must be discarded; in the system that remains it will be necessary 
to discard n—m of the last columns, beginning with the column of 
constant terms (i.e., with the components of the vector C,). The 
last of the remaining columns, composed of the components of the 
vector C,, must be taken as the constant member of the new system. 
The solution of the system gives the coefficients of the linear depen- 
dence of C,, on Co, C,,...,C,,-1, that is, the coefficients of some 
divisor only of the characteristic polynomial. 

We shall here analyse two examples, both taken from the article 
of A. N. Krylov’s [1]. 

As the first example let us determine the coefficients of the char- 
acteristic polynomial of the matrix 


— 5.509882 1.870086 0.422908 0.008814 
0.287865 —11.811654 5.711900 0.058717 
0.049099 4.308033 — 12.970687 0.229326 
0.006235 0.269851 1.397369 —17.596207 


which A. N. Krylov took from Leverrier’s work [1]. By Leverrier’s 
computation, 


y(A) = 24 +47,888430A3 + 797.2789A2 + 5349.457A + 12296.555 = 0; 
A, = —17.86303; A, = —17.15266; As = —7.57404, 
a4 = —5,29870. 
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In Table I we exhibit the scheme for the determination of the 
coefficients p;, regarding them as solutions of system (17’). 

There, in rows | to 4, are arrayed the components of the vectors 
A’Cy (k=0, 1, 2, 3,4) as they are computed, and also the usual check 
sums. (For computational convenience we tabulate the matrix A 
on a separate sheet in transposed form, and obtain the components 
of the vector being computed as the sum of the products of the ele- 
ments of the corresponding column of the tabulated matrix and the 
components of the preceding vector, in one accumulation.) The 
fifth row effects the check, which is analogous to that employed in 
the example of § 17. 


Taste I. Computation of the Coefficients of the Characteristic 
Equation by the Method of A. N. Krylov. Leverrier’s Matrix 





Co Cy Ce Cy Cy = 
7 — 5.509882 30.917951 |— 179.0125! 1100.7201 948.11566 
0} 0.287865 | —4.705449 66.38829 — 967.5973 —905.62659 
0} 0.049099 0.334184 | —23.08728 576.5226 553.81860 
0| 0.006235 0.002224 — 0.649152 — 4.04003 — 4.68072 
— 5.166683 26.548910 |—136 3666) 705.6054 
“|1_ [=16.34603 [| 230.62300 | —3361.2885 | —3146.0115 
“| ‘| 4.136758 | —34.4106¢ | 741.5585 |  708.28462 
0.104141 — 2.087086 16.91760 14.93466 
1 — 30.27086 652.3451 623.0742 
1.065352 — 51.01827 - 40.95081 
| — 47.8887 - 46.8890 
1 — 797.287 - 796.25 
1 — 5349.53 - 5348.40 


(8) 
1 — 12296.8 — 12295.7 
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In rows 6 to 14 is contained the solution of the system obtained; 
we find this solution by the single-division scheme. 

As a final check on the computation of the coefficients p; we have 
a comparison of the value of », with the trace of the matrix. Since 
tr A= —47.888430, we see that the value of p, found from the solu- 
tion of the system is sufficiently exact. 

A comparison with Leverrier’s data shows, however, that the 
coefficients have been computed with a lesser degree of accuracy. 
This loss of accuracy is well-known, and is the inherent shortcoming 
of the A. N. Krylov method; it is to be explained by the circum- 
stance that the coefficients of the system determining the p,; are 
quantities of different orders of magnitude. 

A somewhat better result can be attained by employing the 
pivotal condensation scheme for the solution of the system. 

As the second example we will take yet another matrix from the 
article of A. N. Krylov’s [1]: 


5 30 —48 
(: 14 —24 
3°15 —25 


In the subjoined table the first three rows contain the coefficients 
of the system for the determination of the p;: 





The first step of the Gauss process shows that here we have a case 
of degeneration. The truncated system 


n+5ge = —29 
392 = —15 
gives as the solution g,= —5, z= —4. Thus in this case we have 


determined the coefficients of the second-degree polynomial 
42 4+54+4, which is but a divisor of the characteristic polynomial. 
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§22. THE DETERMINATION OF PROPER 
VECTORS BY THE METHOD OF A. N. 
KRYLOV 


Let us assume that the coefficients of the characteristic poly- 
nomial have been determined by the method of A. N. Krylov, and 
that all the proper numbers have been calculated and have proved 
to be distinct. We shall show how to determine the proper vectors 
of the matrix by utilizing computations already carried out. Let 
Cy be the initial vector in the A. N. Krylov process, and let 
X,, X2,..., X, be the proper vectors of the matrix A belonging to 
Ay, A2,..-,4, In §3 it has been shown that X,,...,X, are 
linearly independent. 

Let us resolve the vector Cy in terms of the proper vectors: 


(1) Co = my X, +0gXy+ --- +a,X,. 
We have, moreover, 


Cc, = ACy => tA, Xy + aylgXy + eee +a,A,X, 
(2) °° hee eta Ee) GE ee a ee a 
Cay = AP Cg = AT 1X, +0,dQ- 1X, + +++ +a,45- 7X, 
The vectors C,, Co,...,C,_,; have been computed in the course of 


finding the proper numbers. Let us form a linear combination of 
them: 


BroCn-1 + BirC,-2 + +++ +81, nC 
= 0 (ByoAT + Bid tt +++ +B, 9-1) Xi 


+ae(Biodg + Buds P+ +++ +81, 1) Xe 
(3) SEL soe sare Desens, tan ee Sees. 


+a, (Bian +BuAn 72+ +: +Bi, 1) X, 


= 0191 (Aq)Xy + 494 (Ag) Xo + Saree +4,91(A,) Xn» 
where 


(4) gi(A) = Biod-1 + By drm? + +++ +81, 1 
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Let us select the coefficients B,9,... , 81, ,-1 80 that 


(5) Pilar) #0, gids) = +--+ = g,(A,) = 0. 
To achieve this it is sufficient to take as (A) the polynomial 
pila) = (A—dg).. . (A—A,) 


(6) = (xd) A=4g) 1 AA) _ (= 19) 
A-A, A-A, 
a (&!)a(4e— pyar“! — ++» = py) 
4-A, 


Here (A) is the characteristic polynomial, whose coefficients and 
roots are already computed. 

The coefficients of the quotient (6) can easily be computed by the 
recurrence formulas 


(7) Bo=1, fi = AsBi, ;-1—Pjs 


j=l,...,2—1 (Horner’s scheme), 


which issue directly from the equation 


(A—A,)(Biod"-! + Bi Ae-2 + ++ + +81, 1) 
= (=I)[an— pit ++ —p. 
Thus 
BroCa-1 +BirCn_-2 t+ + °° +81, w-1Co = 0191(A3)X}, 


that is to say, the linear combination that we formed is the proper 
vector Xj, accurate to a numerical factor. Of course the coefficient 
a, must be different from zero; this is guaranteed by the successful 
completion of the A. N. Krylov process. Since the proper vector is 
determinate but for a constant factor, we may adopt for the proper 
vector the constructed linear combination. 

Analogously 


n-l 
(8) x; = 2 B,jC,-1-}s 
720 
where 
(9) Bio = 1, Bi = AB, j-r—Pjy J=Hl--.,a-l. 


As an example we shall compute, for the Leverrier matrix, the 
proper vector belonging to the proper number A, = — 5.29870. 
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We reproduce the characteristic equation with the coefficients 
taken from Table /: 
A4 + 47,888743 + 797.2B722 + 5349.53 4+ 12296.8 = 0. 
Computing the numbers f,; for 7 =0, 1, 2, 3, we obtain 
1; 42.5900; 571.615; 2320.71. 


We form the linear combinations in accordance with formula (8), 
arranging the computations in the table 


By3Co Bel, By Ce | Boa | X, x, 


2320.71 | — 3149.53 1316.80 | —179.01 308.97 1 
164.548 | —200.405 66.388 30.531 | 0.098815 


28.066 14.233 — 23.087 19.212 0.06218) 
3.5640 0.0947 — 0.6492 3.0095 | 0.009740 





The last column contains the components of the proper vector 
X,, normalized so that its first component equals unity. We shall 
see below that the values of the components of the proper vector 
computed by the method of A. N. Krylov agree well with the values 
computed by other methods. 


§ 23. SAMUELSON’S METHOD 


The method suggested by P. Samuelson [1] is close to the A. N. 
Krylov method. 
The computational scheme of this method is as follows. One 
computes the rectangular matrix 


(1) 
R : 0 0 0 0 ice O 0 ] ay) 
RM 300 0 O ...0 1 =a, —RS 
RM2 : 0 0 0 ... 1 -a,, -RS —RMS |> 


a SS SS 


RMe-1 ? | —ay, —RS —RMS..... 2... —RM=-25 
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where R, S, and M are cells in the following partition of the given 
matrix: 


a $0 yy eee Ay 
: a, R 
(2) A = 491 : Qo9 era aon a ( S M ° 
aut : on 


Furthermore, by means of elementary transformations (as this is 
done in the problem of elimination, § 12) one must attain this situa- 
tion: that in place of the row RM*-! a zero row turns up. Then the 
remaining elements of the last row will give, generally speaking, 
the coefficients of the characteristic polynomial. The process of 
elimination, as we have seen, is very uniform and simple. This is 
the fundamental merit of the scheme. 

The author derives his scheme from a transformation of a system 
of linear differential equations connected with the matrix into one 
differential equation of order n, by means of a special device for 
elimination. A brief algebraic rationale of the scheme is contained 
in the following. 


Let X10 
= X20 = *10 
o e(*) (9 
*n0 


be an arbitrary vector. 
Furthermore, let 
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From the construction it follows that 


(5) Ry = 01941, gr t$RY4A1 
(6) Y, = Sx, pitMY,1 | ee teers 


Thus we have n? relations (n and n(n —1)) between n? +n quantities. 
They make possible the elimination from the system of equations (5) 
and (6) of the vectors Y;,..., Y,, that is, of n(a—1) quantities. 
As the result of this elimination there remain 2 equations connecting 
2n numbers, these being the components of the vector Yp and the 
numbers x19, .- 5 X19 

Let us carry out this elimination. We have, for k=], 2,..., 2: 


Xap = 0yy%1, gr tRYg1 


4) 1%), k-l +RSx,, pot RMY,_2 
= 441%), 4-1 + RSx,, a2 + RMSx,, ppt RM2Y,_ 5 


os © © © © © © © 2 


= 4@1)%), a-1 t+ RSx,, a-g + RMSx;,, 4-3 


SS + RMt-2Sx 9+ RM!A-1Y 5, 
or 


(7) RM#-1Y, SX yy — 24%), 5. 1— RSx,, | ot ae — RM! 28x. 


The coefficients of these n equations obviously form matrix (1). 
Eliminating from these n equations the components of the vector 
Yo, we shall obtain a single lincar relation between the numbers 
X10) -- +» %1, With constant coefficients independent of the choice of 
the initial vector. 
On the other hand, proceeding from the Cayley-Hamilton relation, 
we have 


Sin —P1%), a-1— °° Pak 10 = O. 


This equation is also a relation of linear dependence between the 


numbers x19, . . . » Xj,, With constant coefficients independent of the 
choice of the vector. 


This relation of dependence will coincide with the relation of 
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dependence obtained by the method of climination in case the 
matrix A is such that we may rightly consider the numbers 
X10) + ++» %1, ,-1 to be independent variables, that is, in case we may 
assign them, independently of one another, arbitrary values by 
using a suitable selection of the remaining components of the initial 
vector Xq, or, in other words, by using the vector Yo. 

The Samuelson method may be more rigorously proved by means 
of the following relations between the coefficients of the character- 
istic polynomials of the outer and inner matrices of our bordered 
system. 

Let 


F(A) = (= 1)e[Art part + ++ +64) 


(8) 
P(A) = (Naat tga et +++ +91] 


be the characteristic polynomials of the matrices A and M. (Con- 
travening the usual notational usage for the polynomials, we have 
changed the signs of the coefficients p, and 4,.) 

Then the following relations are valid: 


Py = ay th 


po = —RS— 94011 +92 
bs = —RMS—q,RS— 92011 +93 


Poa = —RM8S— RMS — ++ $953 
f, = —RMS—q,RM-3S— -- - —9,_9RS— 49,1411. 


These relations are obtained from the rule for the expansion of a 
bordered determinant. 

Moreover, if the A. N. Krylov method be applied to the matrix 
M, having adopted R’ (with the components aj, ... , 4),,) for the 
initial vector, the coefficients 9), go, . ~~ 5 J,—-1 Will then be defined 
by the system of equations 


(10) M'=-1R! +q,M'=-2R' +--+ +9,_R’ = 0. 


Tase II. Computation of the Coefficients of the Characteristic 
Equation by Samuelson’s Scheme. Leverrier’s Matrix 


0.422908 
5.208653 
— 183.23653 
3870.8749 
0.226144 | 


1.870086 
— 20.264529 
261.81060 
— 3882.0495 


—242.44343 
4748.777 





0.008814 
0.051697 
— 0.905064 
— 10.72247 
0.004713 
0.147204 
—2.138977 
7.57363 
0.015034 
1.505918 
—63.81948 


5.509882 
0.664047 


47.88902 


—0.559152 
5.577387 
0.534735 

16.346035 
— 140.55844 
2081.445 
1.669435 
264.18511 
— 5846.330 
175.43127 


—0.559152 
0.102131 
30.270872 
— 485.5565 
20.10128 


797.297 | 5349.60 


5.509882 
—0.559152 
5.577387 
—66.36373 
2.946325 
59.146736 
— 765.80173 
11371.416 


6.040710 | 
698.72872 


—17314.569 
463.98856 


12296.94 


8.811690 
—9.053449 
89.197123 
— 76.73268 
4.711917 
86.431330 
~—1144.4327 
18215.162 
8.827310 
995.6906 
—23703.765 
661.185 ? 


(3) 
18492.73 
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On the strength of relations (9), the coefficients p, po, .-.5 Pn 
are non-homogeneous linear forms in q),..., %,—-1 and consequently 
may be calculated simultaneously by the method of elimination (see 
§ 12). Of the two possible modifications of the method of elimina- 
tion, one should take that in which the components of the vectors 
R’, M’R, ..., M’s-1R’, are arrayed in the rows of thescheme. Then 
these rows, conceived as matrices, are R, RM,..., RM). The 
coefficients of the relations (9) will thereupon turn out to be arranged 
so as to accord exactly with the Samuelson scheme. Given the 
grounds for the method that have been presented above, the region 
of its application is easily discerned. Indeed, it coincides with the 
region of application of the A. N. Krylov method for the matrix 
M, proceeding from the vector R’. 

As an example we shall again take the Leverrier matrix. We 
carry through the computation of the coefficients of the character- 
istic polynomial in accordance with the scheme described (see 
Table II). At the outset we compute matrix (1), arranging its 
elements in the first four rows. We next perform the elimination as 
was shown in §12. The last row gives the sought values of the 
coefficients, which as regards accuracy almost coincide with the 
values computed by the method of A. N. Krylov. The last column, 
as usual, is the check column. 

The number of operations necessary to determine the coeflicients 
of the characteristic polynomial by the Samuelson method is some- 
what less than by the A. N. Krylov method, for the formation of 
matrix (1) requires n(n—1)2 multiplications, and the process of 
elimination in the Samuelson scheme requires just as many opera- 
tions as does the solution of the system in the A. N. Krylov method. 
In connection with this we comment that in Wayland’s article [1] 
the reckoning of the number of operations for the Samuelson method 
is incorrectly done. 


§ 24. THE METHOD OF A. M. DANILEVSKY 


An elegant and very efficient method of computing the coeffi- 
cients of the characteristic polynomial has been proposed by A. M. 
Danilevsky [1]. The gist of his method consists in an initial reduc- 
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tion of the secular determinant to the form known as the Frobenius 
normal form: 


Pi-A po fae ee Pn 


l -41 0 0 
D(a) =| 0 I. =3 ; 
0 O:r 80 ne ED 


the expansion of which in powers of 2 demands no labor, since 
D(A) = (=})9[de=—pydet— ++ pr]. 
On the strength of the fact that similar matrices have identical 


characteristic polynomials, in order to attain the aim set, it is 
sufficient to reduce the given matrix 


411 212 +s Ay 


ee © ee ee 


to the form 


1 0... 0 0 


cr 


0 0... I 0 


by means of a similarity transformation. 

We shall show that we can find (n—1) similarity transformations, 
the successive performance of which will realize the desired transi- 
tion from matrix A to matrix P, if this is possible. 

Let us examine the beginning of the process. 

We must carry the row @,; Gy. .-- @q,,-1 4qq into the row 
00... 1 0. Let us assume that a,,,_;40. Divide all elements 
of the row by a, ,_1; then subtract the (n—1)th column, multiplied 
by @q1) Gna) +++» Gng Fespectively, from all the rest of the columns. 
The desired transformation will thereby obviously have been 
effected. 
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The operations we have indicated will thus be elementary trans- 
formations on columns, and, as has been shown in § | Paragraph t0, 
will reduce to a postmultiplication of the matrix A by a matrix M,-, 
which is not hard to jot down, to wit: 





1 0 oes 0 0 
0 l : 0 0 
M,.1 de) nee AEG Sal Se Bing SEN PIE NS ; 
_ 4a Gn9 1 Es Gan 
4q, a1 an, n-1 i Gq, n-1 ay, n-1 
0 0 Sais 0 ] 


The constructed matrix AM,_, will not be similar to the 
matrix A. However from it one can easily pass to a matrix that is 
similar to A. To accomplish this it is enough to premultiply the 
matrix AM,_, by the matrix Mz1,, which is easily computed ; indeed 
one can directly verify that 


0 oO... O l 
The multiplication of the matrix AM,_, on the left by the matrix 
M,_}, obviously does not change the transformed row. 
Thus the matrix 
11 2 CS Poo “15 


Co~1,2 Sn-1,2 +2 Sa-l,n-d Sa-i, 8 
0 0 ee l 0 


has one satisfactory row, and we can continue the process another 
step further if c,_1,,-2#0. If all the (n—1) transformations are 
possible, we shal] have brought the matrix A into form P. 
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An explanation of how one is to proceed in those exceptional 
cases when the necessary elements vanish will be given below. 

At the moment we shall show how the elements of the matrix C, 
the result of one similarity transformation, are computed. 

We have 


a4 a12 es a | MQ, 
AM,-1 = 
4-1,1 %-1,2 > 4-1, n-1 4-1, 1 
an} 4,9 i i | Gan 
1 0 ee 0 0 
x 
My,-1, I My-1, 5 ma-1, n-l1 M,-1, n 
0 0 wate 0 1 
{ on big wee Oy tO 
boy boo tee by, n-1 bo, 
= hr ete ce = B, 
b,-1, 1 b,-1, Qi) s2éne b,-1, n-1 b,-1, n 
0 0 é dee 1 0 
where 


big = Gig +4, p—-1My-1, & for allig n—1, k #n—], 








95 nnd = Fj, p—IMp-1, aod 
Here 
a,; : 
1,5 = -— for i 4 n—-1 
a, n-1 
and 
1 
M,-1, a-1 = a ; 
a, r- 


We see that all the elernents of the matrix B=AM,_,, with the 
exception of the elements of the (1 — 1)th column, are to be computed 
by formulas involving two terms. 

The elements of the (2 — 1)th column, if we recall the value of the 
coefficients m,_1, ,-1,; are obtained as the quotients of the division of 








TaBLE IIT. Computational Scheme for the Method of A, M. Danilevsky 


My M, 


0.006235 
0.269851 
1.397369 
— 17.596207 


My My, 


0.185920 

6.046177 

— 29.461961 
~— 208.45592 


MiY m, 


2.952517 
— 42.32167 
— 562.5575 
— 2244.455 





1 2 
— 5.509882 1.870086 
0.287865 — 11,811654 
0.049099 4.308033 
0.006235 0.269851 
— 0.004462 0.193114 





— 5.511769 
0.262379 — 12.914702 
0.106974 6.812854 


















3 4 z 2’ 
0.422908 0.008814 — 3.208074 


5.711900 0.058717 — 5.753172 
— 12.970687 0.229326 — 8.384229 
1.397369 — 17,596207 — 15,922752 












11.394808 


1.91 a8? 


63.420442 
— 165,46494 


+ canaeataiae '12.592384¢ 





1.610881 
59,332832 
— 156,18272 


0.302646 
4.087610 
9.282220 


5.334234 
71.985155 
— 163.10255 







































0 
6.046177 





“ 16539377 


0.295793 
2.136011 

1 

9 















2.952517 — 42.32167 
Seca 14.33410 
— 1,685430 —79,49874 

1 0 
9 | 
t) 0 
— 47,88843 ~ 797.2789 





1 
— 29,461961 
4.872825 









0 
— 208.45592 
34.47731 





I 
—231.68578 
38.31938 














66.9404 
— 373.2790 
oO 
0 
Y= 9244.455 | 
760.1836 





70.44456 
~ 431.4629 


9.017289 
—58.84347 
0 
1 
~ 562.5575 
190.5349 










































— 4164.769 —5295.913 








~ 12296.55 





—5349.455 ~ 18491.17 
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the elements of the (n—1)th column of the matrix A by the clement 
a 


eae 
1 90 0 0 
0 1 0 0 
C= MI1AM,, =] 000 te 
Qn. Ane Qnon-1 Fan 
0 0 0 ] 
bry bie bint tn 
bay bee be n-1 Om 
See: es. see Ss he NE oe e ae we 
b,-1,1 bn-1, 2 bn-1, 0-1 Sam, a 
0 Oo... ] 0 
bu bis by at b,, 
bar bee be at bo, 


a 


n Rr n 
DX Gqpbir 2D Guten «-- D> 445, nt > SnD gn 
kel kel a) k=l 


0 0 det l 0 


Thus the premultiplication by M,_!, changes only the (n—1)th row 
of the matrix 8; the elements of this row are found with one accumu- 
lation. The entire process, consisting of (n—1) similarity trans- 
formations by means of the matrices M,_1, M,_2,.--, Mj, is fitted 
into a convenient computational scheme, which we shall show in an 
example. 

As the example we again take the Leverricr matrix (Table IIT). 
The Danilevsky transformation consists in the successive multiplica- 
tion of the given matrix on the right by the matrices M3, Mz, and 
M,, and on the left by the matrices M31, Mz!, M;', where the right 
and left multiplications alternate. Each matrix of the transforma- 
tion is given as one row. 
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Let us pass on to the description of the computational process. 
In rows |, 2, 3, 4 are arrayed the elements of the given matrix and 
the check sums. We begin the computation with the computation 
of the elements defining the matrices My and Mj}, viz.: in row 4’ 
we enter the elements of the third row of the matrix M3: 


a4) aye, l G44 
Maye oe MES Ra a. Mak 


443 = 413 

The elements of the third row of the matrix Myz!, equal to the 
elements a,,(k = 1, 2, 3, 4) we enter, for computational convenience, 
in a column: rows 5, 6, 7, 8 of the head-column I. The result of 
the multiplication by M3 we write in the four rows 5, 6, 7,8. The 
transformation M3 brings the fourth row of the matrix into canonical 
form (the 8th row). The elements of the third column are now 
obtained by multiplying the elements of the third column of the 
matrix A by mgg = =~ The rest of the elements are computed by 
the two-term formulas: 

(1) big = Oi +4;gMg,. 

For the check we form the column of sums, as usual. The cle- 
ment of the check column located in row 4’ must coincide with the 
sum of the elements of this row after replacing the element mg by — 1. 
The results of the application of formulas (1) to the elements > are 
entered in the column >’; adding the elements of the 3rd column to 
them, we obtain the check sums for the rows 5 to 8. 

The transformation M;1} changes only the 7th row; we enter the 
result in the 9th row. Its elements are obtained as the sum of the 
products, by pairs, of the elements situated in column I by the cor- 
responding elements of each column of the matrix AM . With this 
we have concluded the transformation M;!AM3. The process is 
continued analogously. We remark that the elements of the 9th row 
are also the elements of the 2nd row of the matrix M 1; we copy 
them columnwise in column I. The elements of the second row of 
the matrix Mz are entered in row 9’. 

The process is easily learned and is afterwards executed without 
difficulty. 

As the result of the computation we obtain all the coefficients at 
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once, since the coincidence—the check—of p, with the trace of the 
matrix is in addition an index to the accuracy of the computation of 
the rest of the coefficients; the results obtained are closer to the 
Leverrier data than are the results found by A. N. Krylov’s method 
or those by the Samuelson method. 

The number of operations necessary for computation by the A. M. 
Danilevsky method is substantially less than by the two other 
methods referred to above. The number of multiplications and 
divisions is equal to (n—1)(n?+n—1). 

We remark that A. M. Danilevsky’s method admits of a modifica- 
tion similar to the pivotal condensation scheme in connection with 
the Gauss method; it somewhat increases the accuracy of the results 
obtained. 

Knowledge of the matrices M,, Mu, ..., M,_, which successively 
effect the similarity transformations, permits us to determine the 
‘proper vectors of the matrix A. 

Indeed, let 2 be some proper number of matrix A and 
X=(x,,...,%,) the proper vector corresponding to it. Let 

Y=(y, yo,-..,¥,) be a proper vector of the matrix P. Then, as 
was shown in § 3 Paragraph 10, 


xX = M,_ 1M, —Beee MuMy,Y. 


Now, the proper vector of the matrix P is found without trouble, 
since its components will be the solutions of the recurrence system: 


(Pi-— A) yi + poyot +--+ +h, = 0 


Yn-1— AY = 0. 


This gives y,=1, y,-)=4,..., yy, = Ar}. 
The transformation M,, performed upon Y, gives 


n 
my, Mygq «++ My 41 > Mus 


o 8 © © © © © © 6 
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The transformation M, thus alters only the first component of the 
vector Y. Analogously, the transformation Me alters the second 
component, etc. 

Thus the components of the vector X are determined by the 
formulas 


k-1 a 
(2) = 3 max,+ > my, (k= 1... 0-1). 


As an example let us consider, for the Leverrier matrix, the com- 
putation of the proper vector belonging to the proper number 4,. 
From the equation 


A4 +.47.8884313 + 797.278942 + 5349.4554 + 12296.55 = O 
we obtain 2,= —5.29872. The computation of the proper vector 


belonging to 4, is performed in conformity with the subjoined table. 


TasLe IV. Computation of a Proper Vector by the 
A. M. Danilevsky Method 


14.33410 . 0.165394 |— 0.193114 28.0764 10.1446 | 0.098822 


190.5349 4.872825 | 0.715631 —5.29872 | 6.38334 | 0.062182 
760.1836 34.47731 | 12.59238 1 1 0.00974) 





Here columns I, II, IIT contain the elements m),, mo, m3,, which 
we transcribe from rows 14’, 9’ and 4’ of Table ITI, setting them up 
in columns. Column IV contains powers of A,, from the third to 
the zero-th. Column V contains the components of the proper 
vector, which are computed one after another by formula (2), which 
is reminiscent of the formula of the Seidel method. Column VI 
contains the components of the vector X4, normalized so that the 
first of its components is equal to unity. 

In concluding this section we shall dwell on exceptional cases. 
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Let us assume that after several steps of the process we have 
arrived at a matrix of the form : 


Cry Cyg cee Cy see Cla 

€g1 C29 Cox C2, 

C= 9 a oke °kn 
0 Oo 1 0 0 

0 60 0 1 0 

0 O 0 #9 1 0 


and together with this it turns out that ¢ ,1=0. 
. Here two cases are conceivable. 

If any one of the elements ¢,; (i<‘—!) is different from zero, we 
interchange the ith and (s—1)th columns and simultaneously 
change the rows with the same numbers. Such a transformation is 
equivalent to a multiplication left and right by a matrix S (see Para- 
graph 10, § 1) of the form 


1 


Onosecvecchiices 1 t 
| : 
S= fo. : 
| PRrerrr erry 0 k—-1 
] 


This matrix has the property that S?=J, and, accordingly, 
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S=S-!, so that a multiplication on left and right by S is a similarity 
transformation. After these transformations one must proceed as 
usual. 

If, however, all ¢,;=0 for i<k—1, then the matter becomes even 
simpler. 

To wit, in this case the matrix C has the form 


Cu C12 (1-1 1k Cin 
€p-1, 1-1, 2+ + + Og-1, g-1 R-1, C-1, 4 CD 
t 
C= 0 O aes 0 ¢, ens ¢ = ’ 
ik ka 0 C 
0 0 0 ] (0) 
0 0 . O 0 ...1 0 
where 
C11 eres C1, k-1 (7) Cha 
] é 0 
C; — > Cy = 
7s a 7 Ss Or os 0 ...1 0 


and consequently 
|C-Al| = |C,—Al]-|C2—Al]. 


The matrix C, already has the Frobenius canonical form and 
therefore |C,—AJ| is computed instantly. In order to expand 
|C, —AJ|, one must apply the general process leading to the matrix P,. 

Thus the case where the process is broken introduces only 
simplifications into the problem of computing the characteristic 
polynomial: an easily computable factor is separated from the 
characteristic polynomial, and the remaining factor is the charac- 
teristic polynomial of a matrix of lower order. 
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§ 25. LEVERRIER’S METHOD IN D. K. 
FADDEEV’S MODIFICATION 


In this section we shall expound a method known as Leverrier’s 
method [2], requiring a greater number of operations than any of 
the methods presented above, but utterly insensitive to the individual 
peculiarities of the matrix, in particular to ‘‘gaps” in the inter- 
mediate determinants. 

Let 


(1) D,(a) = (—1)9[42—p,de-) — podn-2 — --- —p,) 


be the characteristic polynomial of the matrix and 4), An, ..., A, its 
roots, among which some may be equal. Let us employ the symbol 


eo 
1 = G- 


(2) 


Ma 
Mu 


Then a relation is valid that is known as the Newton formula: 
(3) kp, = 5p —PiSp-1— meee —Pi-Siy k=l,...,n. 


If the numbers s, are known, then by solving the recurrence 
system (3) we can find the coefficients p, which we need. 
We shall show how the numbers s, are determined. We have 


5, = Ay tact sivene +A, = trA. 


Moreover, on the strength of Paragraph 11, § 3, the characteristic 


numbers of the matrix 4 will be At, af, ..., ak. Accordingly 
(4) 5, = A+ Ht +--+ +A = Ab 


Thus the process of computation reduces to the successive com- 
putation of the powers of the matrix A, then to the calculation of 
their traces and, finally, to the solution of the recurrence system (3). 
The computation of the n powers of the matrix A (of the last matrix 
A? it is necessary to compute only the diagonal elements) requires a 
great number of operations—uniform, granted—and Leverrier's 
method is inordinately more laborious than the methods expounded 
above. Its value consists, as has already been mentioned, in its 


— 5.509882 
1.870086 
0.422908 
0.008814 


— 232.94165 


— 12296.551 
-0.001 
0.001 

0.001 


0.287865 
— 11.811654 
5.711900 
0.058717 


* 0.000 

= 12296.551 

0.000 

-0.001 
Td 


Taste V. D. K. Faddeev’s Modification of Leverrier’s Method 
Determination of the Coefficients of the Characteristic Polynomial and Elements of the Inverse Matrix 


0.049099 0.006235 4$2.378548 
4.308033 0.269851 0.287865 
— 12,970687 1.397369 0.049099 
0.229326 — (7.596207 0,006235 
— 47.B88430 

564.33711 

9.07995 

— 427.95884 2.68546 


-532.69187 0.300381 
— 797.27876 


2258.1433 
70.5604 
— 4213.8419 32.0618 


— 4649.1712 4.4283 
— 5349.4555 


0.000 — 0.183640 
— 0,000 . — 0.005738 
— 12296.550 - 0.002607 
0.002 — 0.000360 
— 12296.551 


1.870086 
36.076776 
4.308033 
0.269851 


58.98700 
396.31360 
99.69550 
13.01857 


458.3882 
1255,4144 
419.6360 
52.7398 


— 0.037278 
— 0.102095 
= 0.034126 
—0,004289 


0.422908 
5.711900 
34.917743 
1.397369 


23.13088 
132.18346 
369.31992 

25.74858 


276.1613 
556.3836 
1135.6136 
98,8129 


— 0.022158 
— 0.045247 
— 0.092352 
— 0.008036 


0.008814 
0.058717 
0.229326 
30.292223 


0.42522 
2.39755 
4.22567 
264.58689 


6.2599 
11.4757 
16.2164 

700.2843 


— 0.000509 
— 0.000933 
— 0.001319 
— 0.056950 


44.68036 
42.13526 
39.50420 
31.96568 


(6) 
539.9745 


475.9266 
(8) 
301.6549 
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universality. The number of multiplications necessary in the 
Leverrier method is equal to 4(n—1)(2n3 —2n? +n +2). 

We remark that in computing the powers of a matrix it is useful 
to perform a check by means of a column composed of the sums of 
the elements of each row of the matrix 4. The result of the multi- 
plication of the matrix A by this column must coincide with an 
analogous column of the matrix A?, for let >, be the column of 
the sums of matrix A, >» that of the sums of the matrix A?, and let 


U= 
] 
Then: 
21 = AU, By = APU, 
that is, 
(5) 22 = AD. 


What has been said is obviously true for the other powers too. 

We now expound the modification of the method that was pro- 
posed by D. K. Faddeev! which, in addition to simplifying the 
computation of the coefficients of the characteristic polynomial, 
permits us to determine the inverse matrix and the proper vectors 
of the matrix. 

Instead of the sequence 4d, A*,..., 4, let us compute the 
sequence 4,, Ay, ..., A,, constructed in the following manner: 


Ay = A, trd, =q, B, = A,-—qyl 





A, = AB,, == 9, By, = A,—ql 
Mande Bek ee AS Ae ae et 





A, = AB,_1, 


1D. K. Faddeev and I. S. Sominsky (1). 
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We shall show that a) 91:=), go=fo,- ++) In=Pas 
b)  B, is a null matrix; 
c) if Ais a nonsingular matrix, then 
B 


Ari = Sarl, 


(If the matrix A is singular, then (—1)*-B,_, will be the adjoint of 
matrix A.) 
We shall prove a) by the method of mathematical induction. 
That p; =tr A=q, is obvious. We shall assume that 9; =/;, 72= po, 
-+%-1=f%-1, and shall prove that 9,= ),;. In accordance 
with our construction: 


Ay = At—qyAt-1— --- — 4.14 
= At—pAt-l— ... —p, A. 
Accordingly 
tr A, = kq, = tr At—pytr At-!— ... —p,_jtr A 
= 5p —PiSp-r— ++ —Pe-181- 


Hence, on the strength of the Newton formula, kq,=kp,, 9,.=)4- 
Furthermore, on the strength of the Cayley-Hamilton relation: 


(7) B, = A*—pyAvt— --. —p,1 = 0. 


Hence it follows that 4,=,/. This last fact may be utilized for 
checking the computation; it is obvious that the deviation of .4, from 
a scalar matrix is a measure of the accuracy of the computations. 
Besides this final check it is convenient to utilize partial checks by 
forming the sums of the columns of the matrices B;; here the follow- 
ing relation is valid: 


| 1 


(8) Diti = ADi—fit > 21 = 2o-hi , 


] 1 
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where >; is the column of the sums of the matrix B;, and >» the 
analogous column of the matrix A. Finally, from the equation 


1 l 
— AB, =—A, =I 
De De 

it follows that 


(9) A-l = i By. 


Formula (9) provides an algorithm for inverting matrices. For 
matrices of not very high order, in case it is necessary to solve the 
‘problem of finding the proper numbers and that of inverting the 
-matrix as well, this method is very convenient. 

The number of operations necessary for obtaining the coefficients 
_; (including the computation of the matrix B,) is equal to (n—1)n3 
‘multiplications. 

' In Table V is shown the scheme of computation, by D. K. Fad- 
deev’s method, of the coefficients of the characteristic polynomial 
and the elements of the inverse matrix. 

Let us pass now to the determination of the proper vectors of the 
matrix A. 

Let the proper numbers have been computed already, and more- 
over let them have proved to be distinct. Let us construct the 
matrix 


(10) Q, = AU 4+AP AB + +++ +B, 


where 8; are the matrices that have been computed in the process 
of finding the coefficients of the characteristic polynomial, and 4, is 
the kth proper number of the matrix A. 

Tt can be proved that Q, is a non-zero matrix, on the assumption 
that a,,..., A, are all distinct. 

We shall show that each column of the matrix Q, consists of the 
components of the proper vector belonging to the proper number A, 
for, indeed, 


(a,f—A)Q, = (A.J —A)(Ap- + AB, + rich B24) 
= A+ At-1(B, — A) +At-?(By—AB,) + --- +AB,_, 
= Ml—p MU ppl — ++ —4q,1 = 0. 
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Hence it follows that (A,J—A)u=0, where u is any column of the 
constructed matrix Q,, i.e., that 


(11) A,ut = Au, 


which equation shows that u is the proper vector. 

Observation 1. In computing the proper vectors in the manner 
described, there is of course no need to find all columns of the 
matrix Q,. One should limit oneself to the computation of one 
column; its elements are obtained in the form of a linear combina- 
tion of the columns of the matrices B; bearing the same designation, 
with the coefficients previously given. 

Observation 2. For computing the column u of the matrix Q, 
the recurrence formula: 


(12) Up = 6; uy; = Ayu;_1+5,, 


can be used conveniently; here ); is the adopted column of the 


matrix B;, and e¢ is its counterpart column of the unit matrix. 
Then 
“= u,_1. 


As an example let us compute, for Leverrier’s matrix, the proper 
vector that belongs to the proper number A, = — 5.29870. 


TABLE VI. Determination of a Proper Vector by the Method of 
D. K. Faddeev 


I a IV | Vv 
2258.1433 | —2990.2530 | 1189.8295 |— 148.7675 | 308.9523 | 1 


70.5604 — 48.1119 8.0822 0 30.5307 | 0.098820 


32.0618 — 14.2294 1.3785 0 19.2109 | 0.062181 
4.4283 — 1.5939 0.1751 0 3.0095 | 0.009741 





In columns I, II, III are arrayed the components of the first 
column of the matrices B;, multiplied by the corresponding powers 
of 4,; in column IV are the components of the vector A3(1, 0, 0, 0). 
Column V contains the components of the vector X,; column VI, 
those of this vector after normalization. 
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§ 26. THE ESCALATOR METHOD' 


An original method for determining the proper numbers and 
proper vectors of a matrix has recently appeared under the name of 
the escalator method. This method gives an inductive construction 
by means of which, knowing the proper numbers and proper vec- 
tors of the matrix A,_, and its transpose, one can form an equation 
for the determination of the proper numbers of the matrix A,— 
obtained from A,_; by bordering—and next compute, by simple 
formulas, the components of the proper vectors for the matrix A, 
and its transpose. The application of the escalator method is com- 
menced by finding the proper vectors of a matrix of the second order. 
This problem is quite simply soluble. 

The great merit of the method is the presence of a powerful check, 
which makes it possible for the computor to be confident at each 
step, not only of his computations but also of the absence of a loss of 
significant figures. 

The method is based on a utilization of the properties of ortho- 
gonality of the proper vectors of a matrix and its transpose. 

The escalator method is especially convenient in case the complete 
spectrum of the matrix must be found. 

We shall not present the general induction from the kth step to 
the (k+1)th, but shall content ourselves with examining the 
transition from a third-order matrix to a fourth-order one. For 
convenience we shall designate the components of the vectors by 
different letters, contrary to established usage. We shall assume 
that all the proper numbers of the matrix 3 are real and distinct. 

Thus let A, (r=1, 2, 3) be the proper numbers of the matrices 
Ag and A3, where 


(I) Ag = | ao, gq agg 
431 432 433 
Furthermore, let X,=(x,, y,, 2,) and X/=(x', yj, 2’) (r=1, 2, 3) 
be the aggregate of the proper vectors of these matrices. 


2 J. Morris and J. W. Head [1]; J. Morris (2). 
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These proper vectors may be normalized so that 


X, x2 X3 1 41 21 
o é , 
¥i Yo ¥3 %2 Yo Ze 
2; Zg 3 *3 Ys 23 
2 
(2) co , , 
4%) ¥, 2 x, XQ xy 
? o ca 
=] *2 Yo 22 nh ye ys J = ds 
%3 3 «23 Zz 2 23 


This follows from the properties of orthogonality of the proper vec- 
tors of a matrix and of its transpose, established in § 3, Paragraph 12. 

Let A, be the matrix of the fourth order obtained from A3 by 
bordering; let X= (x, y, z, u) be its proper vector which belongs to 
the proper number A. 


We have 
Ax = ay yx +41 0y +4432 + 04 4u 
(3) Ay = 94% + Goo¥ + do3Z + do4t 
Az = ayX + Aggy + dggZ + agg 
Au = aX + agoy + A432 + agg. 


Let us multiply the first three equations of system (3) by 
xi, y!, z;, respectively, and add. We obtain: 


A xx) tyyl + 2Z,) = (011%, + Gary, +4312Z;)% + (10%) + dooy? + A302) y 
+ (443%; + dogy, + 433Z,)Z + (44x) + Goay! +.4342,)U, 


whence, on the strength of the fact that (x), y’, z}) is the proper vec- 
tor for the matrix A3: 


Axx, tyy, + 22Z,) = A, (xx, + yy) + 22Z;) + (a9; + 42qy, + 4342Z,)U 


and consequently 





(4) xx ty, +2z, = — 
where 


(5) Pr = Gy 4X, + do4y; + 434Z,- 
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Let 
(6) P, = 441%, + Gqny, + 4452,. 


Then, on the strength of the orthogonality properties (2), the fol- 
lowing relation is valid: 


”) S Plaietylyt ziz) = P, 

where 

(8) P = agyxtagoyt+aqgz = —(a43—A)u. 
Indeed, 


3 
2, (aai%, + asny, + a4a2,)(4x+4ry t 2,2) 
re 
= gy (4px) +X —Xo + ¥9x3)¥ + Ogi (XY) + X2Yo + X9Y9)¥ 
$41 (X121 +X QZQ+xg2g)Z+ +2 + = Ay X + ayoy +4432. 
Pru 
4,—4 
in accordance with (4), we obtain the following equation for the 
determination of the proper numbers of the matrix A,: 
3 P,P, 

9 ieee Sy rae 

( ) ags A i 1,- A 

Equation (9) we may well call the escalator form of the character- 
istic equation, or the escalator equation. Moreover, on multiplying 
(4) by x,, ¥,, 2, (r=1, 2, 3) successively and adding, we obtain, by 
once again taking into consideration the properties of orthogonality 


(2): 


Replacing the expression x/x+y/y+ z;z in equation (7) by — 

















Fe ee Be oS es. 
uo OSA? ust A,—A’ 
(10) 
ZS Pe 
u ~ rol A,—A 
Analogously 
wo SP OS Pm, 
Ge 2 ge ge eee 
(11) 
z’ 3 Pz’ 
ees > 
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Thus on finding the proper number A from equation (9), we find, 
by formulas (10) and (11), the proper vectors of the matrices A, and 
Ay belonging to this number, accurate to a constant factor. If we 
are to have the possibility of continuing the process, we must still 
normalize them in the sense of formula (2). 

It may be readily verified (again using properties (2)) that 





xx’ -+yy’ + zz) > P,P; 


uu r=) (A,—A)? 
Consequently 
xx! yy! + zz! pu! 3 PP 
un’ aly 2, (A, —A)? 


Thus we will satisfy the normality condition with 
! 3 PP 
a + 2 Rae 


We note that if the escalator form of the characteristic polynomial] 
be denoted by /(A): 


3 PP 
(12) S(A) = -ayytdt+ 3S Tov 
ral A, 
then 
TT ee Rf sie 
PO N+ 2 GRAF 
Thus 


1 ‘ 
= =f'(a). 
Without sacrificing the generality, we may consider that u= +4’, 


choosing the sign so that _ +f’(A) will be positive. This gives 


1 
u=u'= SS>==> if f’(a) > 0; 
(13) vf . 

“w= —y' = Voy if f’(A) < 0. 


The Escalator Method 187 


In concluding we note the check equations 


a) 
* 
I 
Hl 
Ml 


3 3 
ae age . ’ . ’ . 
X, = 4413 2, PY, ayn; > Pz 4435 
a 


3 3 3 
S Pix, =a; > Ply, = ang3 > Plz, = ayy; 


r=) ral rol 


Ay tantagt a, =tr Ag. 


The last equations show that all the “‘new” elements of the matrix 
A, are used by us for the check. A good coincidence of the check 
formulas guarantees the correctyess of the computations at each step. 
Upon conclusion of the process it is useful to verify the fulfilment of 
the orthogonality condition by the vectors that have been found. 

We have described the process only for matrices of the fourth 
order; the transition to the general case is obvious. Given the 
satisfaction of the check equations the method guarantees very great 
accuracy, not only for all the proper numbers, but also for the com- 
ponents of the proper vectors belonging to them. 

In the case of a symmetric matrix, the escalator process is naturally 
facilitated, since all quantities marked by a prime (i-e., relating to 
the transposed matrix) will coincide with the corresponding quan- 
tities without primes. From the form of the escalator equation one 
can conclude that in this case the proper roots of the successively 
bordered matrices separate. This circumstance greatly facilitates 
the determination of the roots, which are usually found by Newton’s 
method. 

We observe that the escalator form of the characteristic equation 
proves to be more convenient to the application of Newton’s method 
than the expanded form, since the computation of f(A) and f’(A) 
is effected very casily. 

Without going into detail, we note that in case successive escalator 
equations have identical or complex roots, the process described 
above must be somewhat modified.! 

We shall find the proper numbers and proper vectors of the 
Leverrier matrix by the escalator method. 


1 J. Morris and J. W. Head [1]. 
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The solution will consist of three stages. 


Stage I. For the matrix Ao, 


— 5.509882 1.870086 \ 
0.287865 —11.811654 


the equation for the determination of the proper numbers will be 


A2 +.17.321536A + 64.542487 = 0; 
its roots: 
A= — 11.895952, Ag = —5,.425584. 


For the check we form 4,+A, = —17.321536, and compute the 
trace of the matrix A2: 


tr Ap = —17.321536. 


Following this we compute the proper vectors of the matrices A, 
and As, solving the corresponding systems and normalizing the 
vectors obtained: 





The first stage has been completed. 


Stage II. We form the matrix 


0.287865 —11.811654 5.711900 


( — 5.509882 1.870086 srs 
0.049099 4.308033 —12.970687 


We copy on a separate sheet the newly introduced coefficients 
433, Ge3 and 43), @3o in the form of columns, and juxtaposing them to 
the columns of the proper vectors of the matrix A,, we find the 
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quantities P; and P; (by accumulation). For convenience in 
future computations we copy them along with the proper vectors in 
scheme (I), arranging them in a row. 

We can now write the escalator equation for the matrix Ag: 


24.124621 0.503193 


S(A) = 12.970687 + 4+ —- aa 11.895959-A* —5.495584—4 


= 0. 


Let us determine its roots by Newton’s method, arranging the com- 
putation in accordance with the scheme: 


—15 | —16.651|}— 17.3458} — 17.3975 |— 17.397655 
— 11.895952—A 3.104 . E 5.501548] 5.501703 
— 5.425584—4 9.574 | 11.225 | 11.9202 | 11.971916| 11.972071 


12.970687+4 |—2.029 |—3.680 |—4.3751 |—4.426813 |— 4.426968 
PPL 
=11.895958— 1 
P,P, 


7.772 5.074 | 4.4267 | 4.385061| 4.384936 


0,053 | 0.045 | 0.0422 | 0.042031] 0.042031 


— 5.425584 —2 


f(a) 5.796 | 1.439 | 0.0938 | 0.000279 }—0.000001 
PP 
(= 11.895952 — a)® 
___P,Py 
(—5.425584— A)" 
La 3.510 | 2.071 | 1.8158 | 1.800570] 1.800525 
Aa -1.651 |—0,6948|—0.0517 |—0.000155| 0.000000 


2.504 1.067 0.8123 0.797059} 0.797014 


0.006 | 0.004 | 0.0035 0.003511) 0.00351! 





Thus A, = —17.397655. 


Analogously we find A, = — 7.594378 and A;= —5.300190. The 
check: A, + Ao+A3 = — 30.292223, 


tr d3 = —30.292223. 


: We now pass on to the determination of the components of the 
‘ proper vectors of the matrices Ay and A3, which are found, accurate 
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to a constant factor, by formulas analogous to formulas (10) 
and (11). 

While so doing it is handy to set up auxiliary schemes (III) and 
(IV). 


Piys Px Pui 
— 1.645356 | 5.618671 |—0.191024| 4.237716 
2.068264 | 0.093231 | 0.240123) 0.070317 
0.422908 | 5.711902 | 0.049099] 4.308033 


1 
Ay + 17.397655 Apt 7.594378 4,+5.300190 
0.181762 — 0.232473 —0.151613 


0.083528 0.461086 — 7.974863 





Scheme (III) contains the corresponding products of the numbers 
P and the components of the vectors, and is obtained from Scheme 
(1); in the last row the check is carried out (for example 


» where for A 





2 ; l 
> Pix;=a13); Scheme (IV) contains the factor tod 
i= cc: 
one takes the three computed roots in succession. 

The normalizing factors are now determined from Scheme (IT) 
and the two analogous schemes serving for the computation of the 
two other roots. Since /’(A;) > 0, 


and z; = z; = D;; t=1, 2,3. 


1 
D, = —_—__ 
"  Vf'(A) 


On computing these, we obtain D,=0.745248, D,y=0.644055, 
Dz =0.172627. 

Utilizing the preceding schemes, we easily find the components 
of the proper vectors of the matrices A3 and Ag (in the final scheme 
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we copy them after multiplication by the corresponding normalizing 
factor). 


x, 
0.094129 | — 0.860553 |2.804268| 0.010928 | — 0.099909 |0.325572 


—0.766896 0.813572 [0.275404 |—0.578409| 0.613612 0.207717 
0.745248 | 0.644055 |0.172627| 0.745248] 0.644055 (0.172627 
0.182847 0.054654 P, and Py 





The second stage has been completed. 
Stage ITI. Having computed the quantities P; and P;, we write 
the escalator equation for the matrix Ay: 


17,596207+A+ ee ae 
+=ymnors0=1 ~° 
and compute its roots: 
A, = —A7.863262, ay = —17.152427, 
As = — 7.574044, A, = — 5.298698, 
5 4, = —47.888431; tr dy = —47,888430. 


We next compute the proper vectors of the matrix A,, normalizing 
them as usual: 


X2 
— 0.019872 0.032932 — 0.351235 1.135218 


0.169807 — 0.261310 0.328467 0.112183 


— 0.187215 0.236640 0.260927 0.070591 
0.808482 0.586694 0.045005 0.011058 
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and the proper vectors of the matrix 44: 
x x: XS 


— 0.014058 0.023297 — 0.248476 0.803091 
0.780383 — 1.200908 1.509559 0.515564 


—~ 1.140762 1.441927 1.589924 0.430123 
0.808482 0.586694 0.045005 0.011058 





The final check of the computation is the computation of the 
product of the two matrices composed respectively of the com- 
ponents of the proper vectors. Instead of the unit matrix there is 
obtained the matrix 


1.000005 —0.000004 0.000000 0.000002 
— 0.000003 1.000004 -—0.000002 —0.000003 
— 0.000002 0.000000 0.999994 0.000000 
— 0.000001 0.000000 0.000004 1.000006 


Lastly, for comparison, we state the escalator equation in the 
ordinary polynomial form: 


A4 + 47.888430A8 + 797.27877A2 + 5349.4556A + 12296.550 = 0, 


and we normalize the proper vector belonging to A, so that its first 
component is equal to unity. This gives X, (1; 0.098820; 0.062183; 
0.009741). 

We sce that the coefficients of the equation here cited coincide 
with Leverrier’s data with greater accuracy than do the computa- 
tions by the other methods. The orthogonality relation is also 
satisfied with considerable accuracy. 


§ 27. THE METHOD OF INTERPOLATION 


The methods that we have presented in the preceding sections 
solved the problem of bringing the secular equation into polynomial 
form. The interpolation method, developed in this section, is 
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applicable to a more general case, to wit, the expansion of a deter- 
minant of the form 


Fis(A) --- Sil) 


(1) F(4) = , 


oe 8 8 © ee ew 


Sala) +++ San(A) 


(fi,(A) is a given polynomial in 4), in particular, to the expansion of 
the characteristic determinant D(A) =|A—AJ| and of the deter- 
minant |A—BA| where A and B are given matrices. 

The essence of the method consists in the following. Let it be 
-known that F(A) is a polynomial of degree not exceeding the number 
-k. As is known from higher algebra, such a polynomial is com- 
‘pletely determined by its values at +1 points and may be recon- 

structed from such values by means of one or another interpolation 
formula. 

For an explicit representation of F(A), therefore, it is necessary to 

compute the value of k+ 1 numerical determinants 


Fila). ++ Saas) 
, TEAiee leon d-ras ss seat co 


Sar(Ai) +++ Sra Ai) 


where do, A;,...,4, are certain numbers chosen arbitrarily, 
generally speaking. 

The computation of the necessary determinants can be accom- 
plished, for instance, by the scheme expounded in § 7. 

For the construction of the polynomial F(A) by its values it is most 
convenient to use the Newton interpolation formula applicable for 
equally spaced abscissas 4,. 

We cite the Newton formula for 4; =i, i=0,.., k: 


(3) F(2) = 2 OOO at)... A-i40), 


=0 
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where AiF(l) designates the ith difference of the computed values of 
the polynomial F(A), which is determined by the recurrence formula 


AiF(l) = Ai-\F(L+1) —4i-1F (1). 
Let us put 


(A-1). -- (Q-i+l) _ eA". 


te mal 


Then formula (3) is transformed into the form 


F(’) = Ds A‘F(0) (2, cai A'F(0)) 
(4) ae 
= FO)+ > ( = caiA'P(0) a. 


This formula bears the appellation “the A. A. Markov interpolation 
formula”. 

In Sh, E. Mikeladze’s work [1] formula (4) has been chosen as the 
interpolation formula. 

We attach a table of the coefficients c,; for m<i<20, expressed 
as decimal fractions.! 

In using the interpolation formula it is convenient as a check of 
the supporting values of determinant (1) to compute one more value 
of F(A), viz., in our case, F(k+1), for 4*+1F(0) must be equal to 
zero, and 4‘F(0) and 4é‘F(1) are equal to each other. 

The interpolation method requires a great many operations. 
Thus for computing the coefficients of the characteristic polynomial 
by means of interpolation formula (4), it is first of all necessary to 
compute (n+1) determinants of the nth order. This requires 
2+ (n—1)(n2-+n+3) multiplications and divisions. If one takes 


the coefficients of the interpolation formula from a table, it is still 
n(n+1) 
2 


necessary to carry out multiplications to obtain the coeffi- 


cients of the characteristic polynomial. 


1 This table was computed by N. M. Terentiev and K. I. Grishmanovskaya. 
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The overall number of multiplication and division operations is 
thus 
n(n+1) 

2 





Ge 1)(n® +243) +—5— 


which exceeds by far the number of operations necessary for com- 
puting the same coefficients by the method of A. M. Danilevsky or 
by that of A. N. Krylov. 

In addition, the method in question does not permit one to 
sin\plify in any way the problem of finding the proper vectors of the 
matrix, whereas in computing by the methods of A. M. Danilevsky 
or A. N. Krylov the task of determining the proper vectors of the 
matrix is much facilitated. Nevertheless the interpolation method 
is interesting as a method making possible the solution of more 
general problems. 

As an example we again exhibit a computation for Leverrier’s 
matrix. 


Omitting the wearisome computation of the determinants, we 
find that 


D(0) = 12296.55, D(1) = 18492.17, D(2) = 26583.68, 
D(3) = 36894.41, D(4) = 49771.69. 


We next compose a table of differences: 


D(A) 
12296.55 
6195.62 


18492.17 1895.89 
8091.51 


26583.68 2219.82 
10310.73 
36894.41 2566.55 


12877.28 
49771.69 





(We note that in case one is computing the coefficients of a char- 
acteristic polynomial, the computation of superfluous values of 
D(A) for a check need not be done, since in this case a reliable check 
is the equality 44F(0) =(—1)#n!). 


WOBNIAN WN ~y 


73 


Taste VII. 
1 2 
1.00000 00000 
—0.50000 00000 0.50000 00000 
0.33333 33333 —0.50000 00000 
—0.25000 00000 0.45833 33333 
0.20000 00000 —0.41666 66667 
—0.16666 66667 0.38055 55556 
0.14285 71429 —0.35000 00000 
—0.12500 00000 0.32410 71429 
OWT LI =—0.30198 41270 
—0.10000 00000 0.28289 68254 
0.090909 09091 —0.26626 98413 


— 0.083333 33333 
0.076923 07692 
—0.071428 57143 
066666 66667 


0: 
—0.062500 00000 
0.058823 52941 
—0,055555 55556 
0.052631 57895 
— 0.050000 00000 


0.25165 64454 
—0.23870 85137 
0.22715 24111 
—0.21677 08218 
0.20738 93121 
—0.19886 64114 
0.19108 62513 
—0.18395 30567 
0.17738 69829 


3 


0.16666 66667 
—0.25000 00000 
0.29166 66667 
—0.31250 00000 
0.32222 22222 
—0.32569 44444 
0.32551 80776 
—0.32316 46825 
0.31950 39683 
—0.31506 77910 
0.31018 99952 
—0.30508 41751 
0.29988 87242 
— 0.29469 38553 
0.28955 82939 
—0.28451 98560 
0.27960 22978 
—0.27481 98358 


4 


0.041666 66667 
— 0.083333 33333 
0.11805 5: 
—0.14583 
0.16788 
—0.18541 
0.19942 
— 0.21067 
0.21974 
—0.22707 
0.23301 
— 0.23781 
0.24169 
— 0.24481 
0.24730 
—0.24925 
0.25077 


99694 
70858 


5 


0.83333 33333- 
56 —0.20833 33333- 
0.34722 ers 


—0.48611 11101 


0.61863 42593. 
—0.74218 75000. 
0.85601 30071. 
—0.96024 16777. 


0.10554 11339 
—0.11422 22865 
0.12214 17270 
—0.12937 15303 
0.13597 89675 


8 —0.14202 54343 


0.14756 62581 
—0.15265 09436 


Table of Coefficients of A. A. Markov’s Interpolation Formula 


10-2 
10-1 
10-2 


10-2 
10-1 
10-1 
10-2 


0.13888 88889. 10-2 
—0.41666 66667-10-2 
0.79861 11111-10-® 
— 0.12500 00000- 10-1 
0.17436 34259. 10-1 
~- 0.22598 37963-10-2 
0.27848 62305- 10-1 
—0.33092 89572-10-3 
0.38267 76988. 10-1 
— 0.43331 40432-10-2 
0.48257 04949. 10-1 
—0.53028 48954. 10-2 
0.57636 84942. 10-2 
—0.62078 35389. 10-1 
0.66352 74910-10-1 
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7 


0.19841 26984 
— 0.69444 44444 
0.15046 29630 
—0.26041 66667 
0.39525 46296 


—0.55063 65741- 


0.72250 00919 
—0.90727 07690 
0.11019 04517 
— 0.13038 56762 


0.15110 24302- 


—0,17216 81227 
0.19344 18265 
—0.21480 89121 


Table of Coefficients of A. A. Markov’s Interpolation Formula—Continued 


8 


0.24801 58730 
—0.99206 34921 


0.23974 86772- 


—0.45469 57672 
0.74618 33113 


0.11123 51190- 


0.15489 69028 
~— 0.20505 51606 
0.26110 82453 
~—0.32244 63934 


0.38847 84994- 


— 0.45864 70640 
0.53243 56241 


9 


0.27557 31922.10-5 


—0.12400 79365 
0.33068 78307 
—0.68204 36508 
0.12035 65917 
—0.19121 33488 
0.28173 03941 
—0.39228 17198 
0.52279 94100 
0.67289 18835 
0.84193 88893 
—0.10291 65477 


“10-4 
10-4 
10-4 
10-3 
10-3 
10-3 
-10-3 
10-3 
10-8 
10-8 
- 10-2 


10 


0.27557 31922 
—0.13778 65961 
0.40187 75720 


— 0.89561 28748- 


0.16913 30467 
—0.28533 30761 
0.44358 12552 


—0.64824 21930- 


0.90267 28545 
—0.12093 17380 
0.15698 20955 


- 10-6 
“10-5 
10-5 
10-5 
- 10-4 
0-4 
10-4 
10-4 
10-4 
10-3 
-10-3 


. 


_ 


0.25052 10839 


—0.13778 65961- 


0.43632 42210 
—0.10448 81687 


0.21027 76553- 


—0.37546 84744 


0.61431 22437- 


—0.94031 83373 
0.13659 18875 
—0.19022 81621 


Za 
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12 


0.20876 75699- 10-8 
—0.12526 05419-10-7 
0.42797 35183- 10-7 
—0.10960 29742.10-8 
0.23417 63229- 10-6 
—0.44126 50535- 10-6 
0.75803 49082 - 10-6 
—0.12130 42722-10-5 
0.18353 50023-10-5 
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13 


0.16059 04384-10-° 
—0.10438 37849- 10-8 
0.38274 05448. 10-8 
— 0.10438 37849- 10-7 
0.23599 43405- 10-7 
— 0.46803 07957- 10-7 
0.84236 33371-10-7 
—0.14067 66531-10-6 


14 


0.11470 74560-10-1° 
— 0.80295 21918- 10-10 
0.31448 96085. 10-9 
—0.91001 24841-10-9 
0.21705 35904- 10-8 
—0.45196 17150-10-& 
0.85054 52978-10-® 


15 


0.76471 63732-1032 
—0.57353 72799-10-12 
0.23897 38666 - 10-10 
—0.73126 00319- 10-10 
0.18351 59980. 10-® 
—0.40032 10556-10-° 


16 


0.47794 77332- 10-13 
—0.38235 81866. 10-12 
0.16887 48657- 10-1! 
—0.54486 04159-10-1! 
0.14351 97385- 10-20 
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17 


0.28114 57254- 10-34 
—0.23897 38666- 10-23 
0.11152 11378. 10-12 
—0.37837 52888- 10-32 


18 


0.15619 20697 - 10-25 
— 0.14057 28627- 10-14 
0.69114 99084. 10-1* 


{9 20 


0.82206 35247-t0-!7 
— 0.78096 03484- 10-16 0.41103 17623-10728 
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Lastly we compute the coefficients of the characteristic poly- 
nomial, arranging the computation in accordance with the scheme: 


A'tD(0) eu 
6195.62 100000000 
1895.89 0.50000000 |—0.50000000 


323.33 0.16666667 |—0.50000000 | 0.33333333 
24.00 | 0.04166667 |—0.25000000 | 0.45833333 |-—0.25000000 
797.280 5349.45 





The coefficient p,=.D(0) = 12296.55. 

The interpolation method is always applicable; in particular the 
case in which the characteristic polynomial has multiple roots is not 
a whit distinguished from the other cases. 

If instead of the numbers 0, . .. , k the numbers A; =a + Ai be taken 
as the interpolation nodes, formula (4) will thereupon be changed 
in form as follows: 


(5) F(a) = Fla)+ ¥ (X eqihid'F(a)) (Qa) 


It may sometimes be convenient to take as interpolation abscissas 
numbers that are not equally spaced. In this case one can use the 
general interpolation formula of Newton’s. However in case of 
abscissas not equally spaced it is more convenient to construct the 
required polynomial by the method of undetermined coefficients, 
viz.: = 

F(A) = agdt +a At“ + - ++ +a. 


Then for the determination of the numbers a,, 7=0,..., &, we will 
obtain a system of algebraic equations 


F(A;) = agAf +a,aFl + +++ +a, 


which may be solved by any of the metheds set forth previously. 
The interpolation method may be employed conveniently, in 
particular, to the expansion of the determinant [4A — BA| in case the 


Comparison of the Methods 201 


matrix B has a small determinant. If, however, the determinant 
of B is not a small number, the coefficients of the desired polynomial 
can better be determined by means of the transformation 


|A—Ba| = |B |AB-1—-Al |. 
The matrix AB-) may be found by the elimination method (§ 12). 


§ 28. COMPARISON OF THE METHODS 


In the foregoing sections there have been expounded six dif- 
ferent methods for bringing the characteristic determinant of the 
matrix into polynomial form; the determination of the proper 
numbers of a matrix is thus reduced to the determination of the 
roots of an algebraic equation. Four of these methods also make 
possible the determination of the proper vectors belonging to the 
proper numbers of the matrix, avoiding the solution of the linear 
systems defining the components of these vectors. 

We shall endeavor to characterize the peculiarities of these 
methods. 

The least number of operations for the entire process of computa- 
tion is required by the method of A. M. Danilevsky. The Danilev- 
sky scheme secures a comparatively high accuracy for the sought-for 
coefficients, which are all determined simultaneously and therefore 
with approximately the same accuracy. The coincidence of the 
coefficient p, with the trace of the matrix serves as a good final check 
of the accuracy. The method makes possible the determination of 
the proper vectors easily enough, too. The somewhat complicated 
“pattern” of the computational scheme is easily mastered by the 
computor. Lastly, the presence of multiple roots of the character- 
istic equation is no obstacle to the use of the method. 

The method of A. N. Krylov requires a somewhat greater volume 
of computations than does that of A. M. Danilevsky. This method 
can nonetheless be recommended for wide application thanks to the 
simplicity and compactness of the computational scheme. 

There is a certain shortcoming of the method: in many cases the 
linear system for the determination of the unknown coefficients 
proves to be inconvenient owing to the coefficients of this system 
being of different orders of magnitude. This may lower the accuracy 
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of the computation. Moreover in case there are multiple roots of 
the characteristic equation, the method permits the determination 
of the coefficients of the minimum polynomial only. 

The Samuelson method differs little in its idea from the A. N. 
Krylov method. Its computational scheme may sometimes be 
useful thanks to the complete uniformity of operations. 

The escalator method solves a somewhat more general problem 
than do any of the rest of the methods described, viz.: we find not 
only all the proper numbers of the matrix, but also all the proper 
vectors of the matrix itself and of its transpose as well, by the very 
nature of the process. The overall number of operations necessary 
for the solution of this problem cannot be reckoned, since in the 
course of the process one has to find the roots of an (n—1)th 
algebraic equation of degree from two to n. And the solution of 
the algebraic equations—which is best carried out by Newton’s 
method—can require a different number of operations in different 
cases. The escalator method is, unconditionally, substantially more 
labor-consuming than are the three methods considered above, but 
it makes it possible to obtain significant accuracy and guarantees 
the reliability of the result. 

The method of Leverrier, and the interpolation method, can 
hardly compete with the first three methods described, for the num- 
ber of operations required for these two methods is substantially 
greater. The value of the interpolation method consists in its 
generality. The method of Leverrier in the modification presented 
here is convenient for matrices of low orders. 


§ 29. DETERMINATION OF THE FIRST PROPER 
NUMBER OF A MATRIX. FIRST CASE 


In connection with problems of mathematical physics it is often 
necessary to find only a few of the first proper numbers of greatest 
modulus, and only the first proper number need be found to a 
significant degree of accuracy. 

To the solution of this problem it is convenient to apply iterative 
methods that in respect of technique recall the iterative methods for 
the solution of a system of linear equations. In using iterative 
methods the dominant proper number is obtained in the form of a 
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limit of some sequence that is constructed by a uniform process. 
The iterative method also gives one the possibility of computing the 
proper vectors belonging to the proper numbers of largest modulus. 

In expounding the iterative methods we shall not consider the 
question in all its generality, but shall limit ourselves to a discussion 
of certain cases that are particularly important in applications. 

In this section we shall assume that the dominant proper number 
of the matrix is real, and that the elementary divisors connected 
with it are linear. For simplicity of exposition we shall assume in 
addition that all the rest of the elementary divisors of the matrix are 
linear, although the derivations that we shall make would hold even 
without this assumption. 

The case we consider is that met most frequently in practical 
problems. In particular, symmetric matrices are subsumed under 
this case, as was established in § 4, as are also matrices all of whose 
proper numbers are distinct. 

On the strength of the linearity of the elementary divisors, to each 
proper number of the matrix A will correspond as many linearly 
independent vectors as its multiplicity. Let 4,, A42,..., A, be the 
proper numbers of the matrix A, arranged in order of diminishing 
modulus (some of them perhaps equal); let X), Xo,..., X, be the 
proper vectors corresponding to them. If all the numbers A; are 
distinct, the vectors X; are determinate but for a constant factor; if 
two of the proper numbers are equal, any linear combination of the 
proper vectors belonging to these numbers will also be a proper 
vector. It is therefore possible so to choose the proper vectors of a 
matrix and its transpose that the orthogonality and normality 
relations (in the sense of § 3, Paragraph 12) are satisfied. 

An arbitrary vector Yo may in this case be represented uniquely 
in the form 


(1) Yo = a,X,+agXo+ +--+ +4,X,; 


where the numbers a; are constants, some perhaps zero. 
Form the sequence of vectors 


(2) AY, A2¥o,..., AtYg,... . 


We shall call the vector A*Y, the fth iterate of the vector Yo by the 
matrix A. 
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Obviously 
(3) AYg = 4 A,X) + aadgXo+ eee +a,A, x 
(4) AtY, = QAR X, + agAhXo+ erets +a,A'X,. 


We use the notation At'Yyo=Y¥,=(yy4, yor, -- >> Yat), and shall 
elucidate the structure of the components of Y,. Let 


Xy = (¥11) Xa1) 0 + ©» Bar) 


Xe = (x12) X29). ++ 5 Xn2)s 


em 6 6 © © © 6 8 ee 


xX, = (19s *ons- ss an) 
We then obtain from (4) 


(5) Yia = AX At tagxjedAst+ +++ +4,x;,A4, 


where the coefficient of Fd reduces to zero when, and only when, 
a,;=Oorx;;=0. The coefficients of Af in all the components of the 
vector Y, can equal zero only if a,=0, since the vector X, is not 
zero. 

Thus any component of the vector Y, depends linearly on 
i, ..., AL Let us denote any of these components by y, (omitting 
the first index). 

Then 


(6) Ye = Ai teadet ++ Hed 
where the coefficients ¢; do not depend on the index k. 
If among the numbers A; some are equal, the corresponding terms 


may be combined. Thus if 4, is a proper number of multiplicity r, 
then 


(7) = cai tea Atay + es +6,A5. 


We shall now assume that the vector Y9 has been so chosen that 
a,#0!1. Then dj appears with coefficients ¢,40 in at least one 
component of the fth iterate of the vector Yo. 

1 Since a;=(X{, Yo), where X, is the first proper vector of the transposed matrix, 


the requirement that a,#0 will be fulfilled in case the vector Yo is not orthogonal 
to the vector X7. 
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Let us consider three cases: 
V1) Jail > [Ae 
2) A= Ag=--- =A; [A] > [Aral 
3) Ay = —ds; Jay] > Asal. 


In the first case we have 


Yar — CaAy* tcgdbt + «++ tegdn 


Yt Cat teodgt «++ +e,ae 
(8) 
<4 1+ boon?) + dyoht +--+ +b ott} 
~ VT bak tp bsokt s+ bck” 
where 
¢; Ai 
(9) b= Pie tae Be 


Carrying out the division and retaining the terms up to the order 
of a2‘ and af inclusive, we obtain 


(10) EE = AL — bya — bya + bob5a3'] + O(a5 +03), 
k 
where 
(11) bs = 6,(l—az), 53 = 63(1 —a). 
Hence we sec that if & is sufficiently large, 
12 a, =~ 4, 
(12) ary 


i.c., the first proper number is approximately equal to the ratio of 
any corresponding components of two neighboring and sufficiently 
high iterates of an arbitrary vector by the matrix A. 

In carrying out the iterations in practice, one should compute 
the ratio 24+! for several components. A good coincidence of these 

r 

ratios will show that in expression (10) the difference of values of 
the coefficients 45, 5g has already ceased to play a significant role. 
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The rapidity of the convergence of the indicated iterative process 
is determined by the magnitude of the ratio z and may be very 
slow. 

Sometimes in computing the iterates it is expedient to divide at 
each step the components of the vectors being iterated by the first or 
largest component, or to normalize them in the customary manner, 
to avoid growth of the components. In doing this, rather than 
the sequence Y, we obtain a sequence 7,=y,A+Yo, where 4, is the 
normalizing factor, and to obtain A; one must take the ratio of the 
components of the vectors AY, and Y,. 

It may occur, although this is improbable, that the initial vector 
Y, has been unfortunately chosen, viz.: such that the coefficient a, 
is equal to zero, or is very close to zero. In this case, at the first 
steps of the iteration the preponderant term will be that dependent 
on A, (if a240). However even if a; is equal to zero exactly, after 
several steps of the iteration the summand dependent on 4,, thanks 
to rounding errors, will appear at first with a very small coefficient; 
in proportion as the iterations proceed, this summand will grow 
with satisfactory rapidity by comparison with the rest. A “struggle 
for dominance” of the terms depending on A, and A, much darkens 
the picture and makes necessary the replacement of the initial vector. 

Example 1. Let us try to determine the first proper number of 
the Leverrier matrix. 

We will take the vector (1, 0, 0, 0) for the initial one and form 
20 iterates, normalizing them at each step. 

We subjoin only the last two iterates: 


Pro AP 9 Ratio of the 
components 
1.00000 — 17.4655 — 17.466 
— 8.20321 143.3809 — 17.479 
8.17013 — 143.0881 — 17.514 
— 7.95957 149.2676 — 18.753 
—6.99265 132.0949 


From the table we see that the ratios of the different components 
are still far from each other; this shows that the process has not yet 
stabilized. Indeed, accurate to three decimals, 4, = — 17.863. The 
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process of iteration converges slowly because of the fact that the 
second proper number A4.= — 17.152 differs little from the first in 
modulus. 

Example 2. Let us determine the first proper number of the 
matrix 


0.287865 ~11.811654 5.711900 


— 5.509882 1.870086 0.422908 
( 0.049099 4.308033 —12.970687 


We shall take (1, 0, 0) as the initial vector. We display a table 
of the iterates commencing with the 12th: 


Pi. AP, Pis AY Pius AP, 
1,0000000 —17.351783 1.0000000 —17.378482 1.0000000 —17.389552 
—8.1139091 141.126754 —8.1332710 141.483894 —8.1413264 141°.632991 
7.8783245 — 137.093170 7.9008117 — 137.468256 7.9102568 — 137.625469 


————— SO 


0.7644154 —13.318201 0.7675407 — 13.362844 0.7689304 — 13.382030 


We find the ratios of the corresponding components for the 12th, 
13th, 14th and 15th iterates to be 


—17.351783  —17.378482 —17.389552 
—17.393189 —17.395694 -—17.396795 
—17.4013!0 —17.399257  —17.398356. 


The three last ratios enable us to consider A, to be —17.39 or 
—17.40. As we have seen (§26), accurate to four figures, 
Ay = —17.3977. 

In case the largest proper number is multiple, we have from (7), 
analogous to the expression we had before, 


Yet _ gy (42)' 
—— = 4,+0(——]- 
% : A 

Thus in this case too, under the condition that 2,40, the ratio 


eg gives the approximate value of the largest proper number. 
k 

The question of the multiplicity of the root cannot be solved without 
a more detailed investigation. We shall return to this question 
again in § 31. 
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Example 3. Let us determine the first proper number of the 
matrix 


1.022551 0.116069 -—0.287028 —0.429969 
0.228401 0.742521 —0.176368 —0.283720 


0.326141 0.097221 0.197209 —0.216487 
0.433864 0.148965 —0.193686 0.006472 


On solving the characteristic equation 


A4 — 1.968753A3 + 1.39118422 —0.4152911+0.044360 = 0, 


we obtain for the proper numbers the values 


A, = dg = 0.667483, Ay = 0.346148, 2, = 0.287639. 


Let us determine 4, by means of the iterative process, taking for 
the initial vector (1, 1, 1, 1). 
We give a table of the iterates beginning with the 9th: 


Po AP, Pro AP io Pi AP, 
1.000000 0.666160 1.000000 0.666822 1.000000 0.667151 
1.844723 =1.230507 =: 1.847165 = 1.232545 = 1.848387 =: 1.233563 
0.676506 0.449420 0.674643 0.449211 0.673660 0.449088 
0.875250 0.583298 0.875613 0.584025 0.875834 0.584399 


4.396479 2.929385 4.397421 2.932603 4.397881 2.934201 


The ratios of the components of these iterates are computed to be 


0.666160 0.666822 0.667151 

0.667042 0.667263 0.667373 
0.664325 0.665850 0.666639 
0.666466 0.666990 0.667249. 


The last four ratios give for 2, the value 0.667, which is correct to 
three places. 

Let us analyse, lastly, the third case, that for which 4, = — do, 
lAy|> As}. 


Determination of First Proper Number—First Case 209 


From equation (6) we sce that in this case the even and the odd 
iterates have different coefficients of corresponding powers of 4, 
since 


(6,400) Az + cgA3t + --- +0,/24 


You 


Yoti = (¢4— 0) UE? + egdght + wes +6,A2tt, 


and two neighboring iterates thus cannot be used for the determina- 
tion of A,;. We can, however, determine A? by one of the following 
formulas: 


+ B Dat 
72 ~ Yet? or Az = Yeet1. 
Y2% Yor-1 


In all the cases we have examined, for the iterative process to be 
successful it is necessary that the first proper number sufficiently 
exceed the one next following, in point of modulus. 

The process that we have described also makes possible the deter- 
mination of all the components of the proper vector belonging to 
the largest proper number, for the ratios of the components of the 
vector Y, tend to the ratios of the components of this proper vector. 

Indeed, for a, 40: 


‘ i 
Y, = ALY, Afar X, +a0(7) Xot wee +a,(7) x,] 
1 1 


& 
aa [x +0(#) x,] 


If A, be a proper number of multiplicity r, the indicated method 
makes it possible to determine one of the proper vectors belonging 
to 4,. We note that in this case, by proceeding from different 
initial vectors, we arrive, generally speaking, at different proper 
vectors. 

In the third case, when 2, = —A,, it is expedient to construct the 
vectors ¥,+A,Y,., and Y,—4,¥,_;. The ratios of the components 
of these vectors will respectively tend to the ratios of the components 
of the vectors X, and X, belonging to the proper numbers A, and Ao. 
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Indeed, in view of the equation 


Y, = a, AX, +49(—A,)'X_+a34§X3+ ote 
we have 


Y,+AYe-3 = 2a, AX, +03(Agt+A,)Ag 1X3+ ean 


~ fea) 


Y,—Ay¥y_1 = 2ag(—A)*X_+49(Ag—Ai)Ag 1 Xa+ ++ 
k 
e (—A)4(200Xe + o( #*) ) 


As an example let us find the proper vector belonging to the first 
proper number of the matrix of Example 2. In § 26 we determined 
the components of this vector, viz.: 


X, = (0.094129, —0.766896, 0.745248) 
or, after normalizing, 
X, = (1.00000, — 8.14729, 7.91730). 


From the table exhibited in Example 2, we find for the com- 
ponents of the vector the following values: 
1.00000 1.00000 1.00000 
—8.13327 —8.14133 —8.14472 
7.90081 7.91026 7.91426 


We see that the last result already approximates closely enough 
the exact value. 


In concluding this section we shall find the first proper number, 
and the proper vector belonging to it, of the matrix 
0.22 0.02 0.12 0.14 
0.02 0.14 0.04 —0.06 
0.12 0.04 0.28 0.08 
0.14 -—0.06 0.08 0.26 
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In Table XIII, § 17, 14 iterates were computed, starting with the 
vector (0.76; 0.08; 1.12; 0.68). 

Computing the ratios of the components of the 14th and 13th 
iterates, we find for A, the value 0.4800. 

(We ignore the second component of the iterates because of its 
smallness by comparison with the other components.) 

The ratios of the components of the 7th and 6th iterates give for 
A, the values 


0.4800 0.4792 0.4808. 


For the components of the proper vector we find from the 14th 
iterate the following values: 


(1.0000 0.0000 1.0000 1.0000). 


It is readily verified that the exact value of A, =0.48, and that the 
proper vector belonging to it has the components (1, 0, 1, 1). 


§ 30. IMPROVING THE CONVERGENCE OF 
THE ITERATIVE PROCESS 


The convergence of the iterative process may be very slow. 
The cause of this is sometimes the presence of a nonlinear elementary 
divisor corresponding to the first proper number. In § 34 it will be 
shown how to reveal this circumstance and how to proceed in such 
acase. But the cause of the slow convergence may also be the close- 
ness of the second proper number to the first. In this case the con- 
vergence may be improved by means of certain devices. 

1. Raising the matrix to a power. It is sometimes expedient to apply 
Lobachevsky’s method to the characteristic equation of the matrix. 
This can be done without finding the characteristic equation in 
explicit form, in the following manner. Successive powers of the 
matrix A: A2, At, A8, A!6,... are computed. For each of these 
matrices the trace is computed, and then A, is determined approxi- 
mately by the formula 


(1) Am A/ tr A®, 
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The last formula follows from the fact that 
tr A” = AT+ A+ +: + AR, 


and consequently 


Vir dm = 2, Lis(By +P 


m 
~ sof) 

Instead of extracting the root one can begin the usual iteration of 
an arbitrary vector, using the constructed powers, so that knowing, 
for instance A8, A8Yo can be easily constructed, and then A®(A® Yo) 
=Al6Y, and finally, A!7¥9=A(A!®Yo), and A, then found as the 
ratio of the components of the vectors 417¥q and A18¥,. 

Obtaining each power of a matrix requires, however, n3 opera- 
tions, and is therefore equivalent ton iterations of the vector. If the 
convergence of the process of iterations is good, sufficient accuracy 
in the determination of A, will be attained after a few steps and there 
will be no need to have recourse to the process described. One 
should turn to it, however, if the number of iterations for attaining 
the requisite accuracy exceeds n- logon. 

In computing the powers of matrices one should use the check 
described in § 25. 

2. The scalar product. This device is particularly handy in 
application to a symmetric matrix; we will expound it here without 
that assumption, however. 

Together with the sequence of iterates of the vector Yy by the 
matrix A, 


(2) 


(3) Yo, Y; = AYo, Yo = A2Vo,..., Y, = A*Y¥o,.. 


let there also be computed the sequence of iterates by the matrix 4’, 
the transpose of A, 


(4) ‘Yo, ¥, = A’Yp, Vn = A’2Y,..., Vi = A’EYg,... 


Let 5,, ..., 5, be the coordinates of the vector Yo with respect to 
the basis Xj,..., %X,; and a,,..., 4, the coordinates of Yo with 
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respect to the basis X,,..., X,. We assume in addition that the 
bases are so selected that the system of vectors X,, Xs,..., X, and 
Xj,..., Xp satisfies the conditions of orthogonality and normality 
in the sense of § 3 Paragraph 12. 

Let us form the scalar product (Yj, Y,): 


(Yi, Vy) = (A’*Y9, At¥o) = (Yo, A2*Y¥o) 
= (bX) + boXot ++ - +8,X5, 
a,AP'X, + aghAtXn+ --- +a,A4X). 


A further step is made possible by the properties of orthogonality and 
normality of the system of vectors X,,..., X, and X,..., Xq: 


(5) (Yi, Vy) = 01d, AF + cable + ++ +4,b,274, 
Analogously, 
(62) (Via Ya) = ay AP tagbodg + --. +0,b,atO. 


From equations (5) and (6) we obtain 


(Yin Ys) yb AF tagdodZ + «+ - +anb,A2 
(7) 
Ag\ 2 
= a+0(32) 


From this estimate it is evident that the formation of the scalar 
product lessens the number of steps of the iteration required for the 
determination of A, to a given accuracy by almost half. However 
along with this, sequence (4) must be computed in addition. 

In case of a symmetric matrix, sequences (3) and (4) coincide, and 
thus the application of the scalar product method is particularly 
efficient. Beginning with a certain step of the iterations, one must 
compute the corresponding scalar products and determine 4, by 
their ratio, to wit: 


(AtYo, A*Yo) 


A = (AFT, APY) 
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Example. Let us consider the matrix 


1.0000000 0 1.0000000 0 

1,0000000 0.7777778 0.3333333 0.3333333 

0 ~— 0.0252525 0.5555556 -—0.0252525 |: 
0 ~—0.8888889 —8.6444444 O.L1NI 


whose proper numbers are, 1, 2, # and 4. 
For the determination of A, we form the iterates A‘Yo, taking as 
Yo the vector (1, 1, 1, 1). 

AlTY, ABy, AY, A®Y, 
4.6731097 4.6760089 4.6779433 4.6792336 
8.3733415 8.3912886 8.4032694 8.4112637 
0.0028992 0.0019344 0.0012903 0.0008605 

—8.3861607 —8.3998278 —8.4089592 —8.4150555 


4.6631897 4.6694041 4.6735438 4.6763023 


The last row maintains a check of the computations. 
The ratios of the three corresponding components of these 


iterates will be 
1.000414 1.000276 


1.001428 1.000951 
1.001087 1.000725. 


The last column gives 1,~ 1.001; the value found coincides with 
the exact one accurately to one unit in the third place. 
We shall now show how this value can be refined by an applica- 
tion of the scalar product method. 
With this in view we form iterates of the vector (1, 1, 1, 1) by the 
transposed matrix A’. 
On computing them, we obtain 
A’20¥q = (0.7961118, —0.0002189, 3.9939022, —0.1134904). 
Furthermore 
(A20Yo, A’20¥,) = 4.681817 and (A!®¥q, A’20Y9) = 4.681816. 
The ratio 
(A®°Yo, A’2°¥9) 
(Alo, A’2°Y5) 


gives A, the value 1.000000, correct to six decimals. 


= 1.000000 
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Observation. If in finding the iterates we normalize them, then 


_ (AY, 1, A*¥o) 


7 
: (Y,-1, A’*Yo) 


where normalization of vectors A’*Yo is a matter of indifference. 

3. The 62 process.1 This device is applicable only in case 
|A,|>|Ag|, and A), Ag and Ag are real. 

Let us assume that we have determined a number of quantities 


(8) Yas Yaris Yetar soos 


about which it is known that 
(9) ty = db tegdgt --- +044. 


As y, one can take, for instance, any component of the vector 
Y,=A‘Yo, the trace of the matrix A‘, the scalar product of corres- 
ponding iterates, etc. Then, as has been shown in § 29 and § 30, 
the first proper number A; can be approximately determined as the 


ratio m4 = get 
4 


Moreover it was shown in § 29 that 
(10) u, = A,[1 —bjoh — byak +.b,b5024] + O(a! +034), 
where 


bn ip aay a 
aaa (l @»), era a3) and a ee 


If the convergence of the sequence 1, U4), %j+2,--- is insuffi- 
ciently rapid, it can be greatly improved by the following device, 
which is called “‘the 6? process”’. 

Let us form 


uy Ug+1 








Ug+i Ueto] 
Hy — Quysi t tyte 


(11) P(y) = 


1 A. Aitken [1]. 
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We shall show that 


(12) P(u) = 4,+0(7 +0(#)" 


With this aim in view we put 


uy = Ay(1+ &), 
then 
allte,  I+eail, 
lt epy 1+ eee 


u, Ug+1 
Ugty Upto 











Separating the last determinant into the sum of four, we obtain 














u u & 
k ret] 32 [es Pesseued ht ] 
Ut. Upto nei Eg+2 
But 
Ug — Qugay + ttgee = Ay leg — Zep t+ pte). 
Thus 


& Eps 
Egt1 Ete . 
&,—2Eg41 + Egte 








P(u,) = a) + 


On calculating we find that 





& gti 
Egt1 — Exte Act + Baz 
enti 3 
where 
— b3(a2—ag)? 1.8 
A= — Trae 7 B= baba 
Thus 


re = vo) 2) 


Hence it follows that the error in the determination of A, with the 
aid of the 6? process may be less by far than in a direct determination 
of it from the sequence 1, uz44,.-- 

Observation. In finding the first proper number of a symmetric 
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matrix, the 62 process must be applied not to the components of the 
iterates, but to corresponding scalar products. 

Lastly we note a property of the operation P(u,) that is convenient 
for practical computations, viz. : 


(13) P(uyte) = c+P(uy). 


Hence it follows that before employing the 6? process, one can deduct 
a convenient constant from the numbers 44, u+1, %+2. (Such a 
constant could be, for example, one consisting of the number of 
decimals that have already been established.) 

We shall show the application of the 6° process to the examples of 
§ 29. 

In Example 2, § 29, for the ratios displayed the application of the 
6° process gives for A, the following values: 


— 17.3974 
— 17.3977 
~ 17.3977. 


Accurate to four decimals, 4, = — 17.3977. 
Analogously, in Example 3, § 29, the application of the 6° process 
to the displayed ratios gives for 2, the following values: 


0.66748 
0.66748 
0.66748 
0.66748. 


Thus A, is already determined with an accuracy of five decimals (the 
exact value of 1, = 0.667483). 

The 62 process may also be applied to the determination of the 
components of the first proper vector. In this connection we shall 
show two distinct variants of this process, the choice depending on 
whether 4, can be considered as known to a sufficient degree of 
accuracy or not. 

1) Let there be known to us only the sequence of iterates A‘Yo, 
where for computational convenience let each iterate be normalized 
by division by a fixed component z,. 

We shall denote any other component of the vector Y, by y,. 
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If 
t k k 
Yg = CAL teodat +++ tenda 
y= by A} + bods + dy + b,Ak, 
then the indicated division reduces z, to unity and y, to v,, where 


_ Me A teed t +++ tends 
Ze by AE + bog + «++ +b,ah 

ere tld ae) (3)’ 

S50 at. tay 

A simple computation shows that 

a zy (3). 
Poo) = 5 +0(z) +0(% 

Thus if the 6? process is applied to all components of the normal- 
ized vector AtY, we shall find the ratios of the coefficients standing 
with the powers of A, in the expressions for y,. These coefficients, 
moreover, are proportional to the components of the proper vector. 


Thus for the proper vector of Example 2 of the preceding section, 
application of the 62 process gives for the components of the vector 


the values 
1.00000 
—8.14718 }> 
7.91721 


which are considerably closer to the exact ones than the values com- 
puted directly from the same iterates. 

2) If A, is known with sufficient exactitude, an improving process 
can be constructed in the following manner. We multiply all the 
components of the vectors At—¥o, AtYo, Att+1¥o, by ay, 1, Ay? 
respectively, and then apply the 62 process to them. Since 


Mardy = Mey tAy Aye + +++ Ae Aye, 


94 = Mert Aste t +++ + Ante 

et ‘ ae k+1 
Yeerdy = Ae + F—egt ++ + oy 
Ay A 
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we have 
Ae ~1ak~ 
a Wi ee = [eycody An (Ay — An)? + + - 
toad ae a, ayelfi+o(2)]. 
1 
Moreover 


k-1 k-1 
Aaya 2yet A "'Yar = A (Ay—Ag)P+ +++ +e, x (A, —A,)?. 


Hence 
| Aya-1 % 


ye AL yet 3 [ 1 (2) 
= + Of — i 
Ye-1 — 24g + Yet an A, 


The ratios of the numbers obtained, which latter are computed for 
the different components, give the ratios of the components of the 
proper vector. 


§31. FINDING THE PROPER NUMBERS 
NEXT IN LINE 


In this section we shall show that by modifying the process of 
iteration in a certain manner, one can determine the product 
of several proper numbers. This opens up for us the possibility of 
determining one after another the proper numbers next in order 
after the first. It should be remarked, however, that the deter- 
mination of the product of even two of the proper numbers en- 
counters great difficulties, chief of which is the disappearance of 
significant figures. As a rule, therefore, even the second proper 
number determined by means of the iterative process is of a much 
lesser degrce of accuracy than the first. 


Let 

& k t 
(1) y= 6A; +eodg+ aoe +¢,45 
zy = dt +-dodb + sexe +d,A4, 


be any components of the vectors AtYg and AtZp. (If Zo= Yo, 
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y, and z, will be any two components whatever of the vector .4*Yy.) 
Let us form the expression 


Ye Yer 
Zp 2g+1 


. 


(2) [Yes Ze] = 








On working this out, we find that 
(3) [Ye Ze) = (Cod —¢dy)(2y— Ag) Ai Ae 


+ (Cyd —ydy)(Ay — Ag) ALAS + O(A2A8). 
Hence it follows that 


(yeti, Z+1] zy 
4 Te A de O(—)}- 
(4) [¥e> 2;] fet (3 


In particular, one may take Z>=AYo, whereupon 


Ye Yeti 
Yet1 Yeto 








(5) [ys Ze] = [Ye Yt] = 
It can be verified that 


(6) [Yes Yeti) = 61¢0(1 — Ae) 2A, AS + €169(Ay — Ag) 2A} 5 + 0( AAG). 
Therefore 


[yit1, ¥ite] (2) 
7 a = 1,204 0( =] - 
(7) send NG, 


The disappearance of significant figures which we alluded to is 
occasioned by the fact that the elements of determinant (2) and of 
determinant (5) as well become almost proportional to one another 
to the degree that the step of the iteration is increased. 

Observation, In case the first proper number is multiple, 
A, =4,= +--+ =A,, but the elementary divisors corresponding to it are 
linear, the process indicated gives the product 4,4,4,. This process 
will generally give the product of the two “eldest” roots of the 
minimum, and not of the characteristic, polynomial. Therefore in 
utilizing the iterative process, one can judge of the multiplicity of 
the dominant proper number only by means of indirect considera- 
tions. An indication of the order of the multiplicity can be given, 
for example, by a comparison of the trace of the matrix with the 
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computed magnitudes of the first and second roots of the minimum 
polynomial. 

It is obvious that by means of three components of the vector 
A‘Y, or by means of corresponding components of the vectors 
ALY, Ak+1 Yq, Ak+2¥o, we can analogously construct determinants 
[Yes Ze. 4) or [¥n, Yeti, Yeto]. The ratios of corresponding deter- 
minants will give us an approximate value for the product 1,12A3. 
The process can, theoretically, be continued to the determination of 
the product of the k roots. Practically, the process yields only very 
rough values for the product of two or three roots. 

As an example we shall determine the second proper number for 
the matrix we examined in Paragraph 2, § 30. 

Adopting for y, the second component of the vector A‘Xo, and for 
z, the fourth component, we obtain 


8.3912886 8.4032694 


[vss 218] =| _g 3998978 —8.4089592 | ~ °-0240124. 


Analogously 
[y¥19. Z19] = 0.0159949, 


Thus 2,A4,~ 0.666110. 

In § 30 we saw that A, 21.001. This gives the value A. = 0.6654. 
If we use the value refined by the scalar product method, 
A, = 1.000000, we will obtain as 2. the value 0.666, which is correct 
to an accuracy of one unit in the third decimal: 


(ay = 3 = 0.66666... ). 


If as y, the first component of the vector A*Xq be adopted, and the 
second component as z,, we will obtain 


[yg 218] = 0.0397902 
[y19) Z19] = 0.0265541. 
This gives 
AyAo ~ 0.667353. 


Utilizing the refined value of 4), we obtain 2.~0.667. Finally 
we shall determine A, by the ratio of determinants of the form 


Ye Yet) 
Yeti Yate 
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where y, is any component of the vector A*Xp. Taking as y, the 
first component, we obtain 


4.6760089 4.6779433 
[ye Yio] = | 46779433 4.6792336 | ~ — 2-0030156. 


Analogously, 
[¥175 ¥is) = —0.0045170. 


Thus 2,4.0.66761 and 2.~0.668. 


§32. DETERMINATION OF THE PROPER 
NUMBERS NEXT IN LINE AND THEIR 
PROPER VECTORS AS WELL 


The process of determining the proper numbers following after 
A, described by us in the preceding section, does not make possible 
the determination of the proper vectors belonging to these numbers. 
In this section we shall give devices that permit the determination 
not only of the proper numbers that follow A), but also of the proper 
vectors belonging to them. 
1. The A-difference. Let there be computed the sequence 


(1) Ys Yor eo Y¥imv ser Yere ee 


and from it let A, ate be determined. Here y, is any component 


k 
of the iterate Y,=A*Yo. 
Let us form the difference 


(2) Yer — Ay, = €9(Ag— a1) A5 + see +6q(Aq— Ay) Ane 


If A, is greater in modulus than all the rest of the proper numbers 
and ¢.#0, the first term of this difference will predominate and we 
shall be able to determine A, in a way analogous to that by which we 
determined 4), viz.: 


(3) de = Yet —Ay, 

Ye—ArYe-1 
However in such a determination of A, we encounter a disappearance 
of significant figures, since in the numerator and denominator of 
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ratio (3) we have to subtract quantities close to one another. It is 
expedient in practice, after finding A, from the ratio of y,4, and y,, 
to turn back and determine A. from the ratio 


(4) Ag ~ Ymt) Ayn 


m<k, 
Ya AyYm—1 


taking as m the least number for which the predominance of A, over 
the succeeding proper numbers has already begun to make itself 
felt. The suggested device gives for A, values that are rough, to be 
sure; however they are frequently adequate for practical purposes. 
Theoretically it is possible by means of an analogous process to 
determine the succeeding proper numbers too. 

It is obvious that to determine the second proper vector the process 
of forming the A-difference must be carried out on the sequence 
AY , A?¥o,..., AtYo, ...; indeed, the difference 


ANY, —AsA*¥y = ag(Ag— Ar) Ket ++ +$0,(Ay—Ay)aEX, 


shows that the components of the vector X, may be found in a 
manner analogous to that with which we determined the com- 
ponents of the vector X, in § 29. 

For an example, we determine the second proper number of the 
matrix discussed in § 30, Paragraph 2. 

As A, we shall take not only the value obtained directly from the 
ratios of the components of the 20th and 19th iterates (A,;~ 1.001), 
but also the value refined by means of the scalar product 
(A, = 1.000000). 

Adopting as y, the first component of the vector A‘ Yo, we obtain 
(for A;~ 1.000000), taking into account the 17th, 18th and 19th 
iterates (m= 18): 


Yut1—AiYn _ 4.677943 — 4.676009 _ 0.001934 _ 9 gaa. 


ae = gn ~ 4.676009 —4.673110 ~ 0.002899 


Analogously, adopting as y, the fourth component of the vector 
A‘Yo, we obtain 


—0.009131 
ag = =0.013667 ~ 0.6681. 
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Thus knowledge of a sufficiently exact value of A, has made it 
possible to determine 4, also with sufficient accuracy (to three places 
after the decimal). (Exactly, A4,=0.666... .) 

If as A, we take the rougher value 2,~ 1.001, on computing the 
previous ratio we run up against the phenomenon described—the 
disappearance of significant figures. In this case as m a number 
considerably less than 20 must be taken. 

Thus, considering the 9th, 10th and 11th iterates of the vector 
AtYo, 

A®Y, Al0Y, Ally, 
4.4665336  4.5365193  4.5841480 
7.1243407 7.5407651 7.8281626 
0.0699857  0.0476287 0.0321941 


—7.4707539 —7.7678185 —7.9777169 
4.1901061 4.3570946 4.4667878, 


we obtain, on computing the values y,,41—Ajy,: 
m=9 m=10 
0.06552 0.04309 
0.40930 0.27986 
—0.02242 —0.01548 
—0,.28959 -0.20213. 


The ratios of these quantitites give for 2, the values 


0.658 
0.684 
0.690 
0.698. 


Thus knowledge of a value for A, that is very rough allows us none- 
theless to obtain for A, a value accurate to three units of the second 
place, by using the early iterates. 

The A-difference method permits the determination of the com- 
ponents of the second proper vector too. 
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For the matrix under consideration, the second proper vector has 
the components 


(1, 3 = 6.2, -3 = —0.333..., -7} = —4.733...). 


Approximate values for the components of the second proper 
vector can be obtained as the respective ratios of the components of 
the vector Amt! Yy—A,A"Yp. Taking m=9, we obtain, by utilizing 
the components of the vector 4!°Y,—4,A®¥o, computed earlier, the 
following values for the components of the proper vector (after 
normalization) : 


(1.00; 6.49; —0.36; —4.69), 


which is in sufficiently close agreement with the exact value. 

2. The method of exhaustion. The method of exhaustion in the 
general casc requires not only knowledge of the first proper number, 
but also of the proper vectors that correspond to it for the matrix A 
and its transpose as well. 

Let us say that we have determined with sufficient accuracy 4), 
X, and X{, where the vectors X; and Xj have been so normalized 
that their scalar product is equal to unity. 

Regarding the components of the vector X, as a column, and 
the components of the vector X; as a row, we form the matrix 
product X,X}. This will be the square matrix 


7 a. s ° 
CO tS 6 MS Dos) MCS B Sal 5 
? 2 ¢ 
Xai%11 Xeiay +++ XaiXny 
- 4 ¢ 
SAi%11 XpiX2q0 os Kg itt 


We remark that the matrix product XX, is a number, to wit the 
scalar product of the vectors X; and X). 
Next form the matrix 


(5) Ay = A-A,X, 1 


We shall show that the matrix A, has the same proper numbers 
and vectors as the matrix A, with the exception of the first proper 


226 Proper Numbers and Proper Vectors of a Matrix 


number, in place of which there will be a proper number equal to 
zero. Indeed, 


AyX, = AX, —A(X,X4) Xi 


= AX, — A,Xy(X1X)) = AX, —A,X, =0 
(6) 
AyX; = AX;—Ay(X1X1) X; 


= AX;-1,X\(X1X;) = 4X, (G41) 


since X}X,=1, and X}X;=0, on the strength of the orthogonal 
properties of the vectors X,,..., X, and X},..., Xn 

The property of the matrix A, which we have indicated makes it 
possible to start from the vector sequence A, Yo, ..., A,"¥o,... and 
determine 4, and X, in a manner analogous to that by which we 
determined A, and X, from the sequence AYp,..., A#Yo,..., since 
the proper number A, will be the first proper number of the matrix 
A,. We shall call this process the ‘‘ process of exhaustion”. 

We shall show that 


(7) ATYg = A"Yo— A X1X1¥o, 


i.e., that in the practical application of the indicated process there 
is no need of computing the matrix A, in fact, or of forming the 
series of vectors A,Yo,...,4,"Yo,..., but that it is sufficient to 
compute just two adjacent vectors 4,"+!Y, and A,"Yo by formula 
(7). 

In order to establish equation (7) we shall introduce the so-called 
bilinear resolution of the matrix A. 

On the basis of the orthonormal properties of the system of proper 
vectors of the matrix A and its transpose, the following matrix 
equation is valid: 


PENNA wt AIG 


Multiplying this equation on the left by A and replacing 4X, by 
4,X;, i=1,..., 2, we obtain 


(8) A = AX X41 +AX XO + +++ +A, X, Xa. 
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The process of exhaustion annihilates the first sammand in this 
resolution, whence it follows that 


Ay = AgXoXat +++ tA X,Xn 
Moreover 
A” = ATX X FAX Xa+ 0 FAX, XG. 
Analogously 
AT = ABXeXa + +++ + APX,X,. 
Therefore 
AT = A"— AX, X4, 


whence issues equation (7). 

Thus for the practical application of the process of exhaustion, 
one has to compute the vector X,X1Yo, form the vectors 4,"*'¥, 
and 4,"¥o by formula (7), and then find A, and X. in a manner 
analogous to that by which we determined A, and X, in § 29. It 
is obvious that in course of this all the devices for improving the con- 
vergence of the iterations which we have described in § 30 may be 
employed, in particular the 6? process. 

We note that one must take as ma number considerably less than 
the number of the iterate from which the components of the first 
proper vector are determined. We note too that in case one finds 
it necessary to determine with great accuracy the second proper 
number and the vector belonging to it, one will find it in fact 
expedient to form the matrix A, and to compute its iterates. 

Example. Let us again consider the matrix of § 30, Paragraph 2. 

The method of exhaustion requires, for the determination of the 
second proper number, knowledge of the first proper number, as 
also of the proper vectors of the matrix A and of matrix A’ as well 
that belong to it. It is therefore necessary in utilizing this method 
to compute, along with the sequence of iterates A‘ Yo, that of AY 
also. Thus in determining 4, we must always refine the value 
obtained , by means of the scalar product method. 

In our example, using twelve iterations of the vector Yo 
=(1, 1, 1, 1) by the matrix A and by A’, we have obtained for 4; 
the value 2, = 1.000000 (see § 30, Paragraph 2). 

For the components of the proper vectors of the matrices A and A’ 
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we obtain, after normalizing the components of the vectors .42°Y 
and A’20Y), the values 


1.00000 1.00000 

1.79757 —0.00027 

0.00018 5.01676 

— 1.79838 —0.14256. 
(The exact values of the components of the first proper vector of the 
matrix 4 are (1, 1.8,0, —1.8).) Following the theory, it is first 
necessary to normalize the vectors X, and Xj so that (X;, X})=1. 
The normalizing factor computes out to be ¢=0.795678, Thus for 


the components of the first proper vectors of the matrices A and A’ 
we obtain the values 


1.00000 0.79568 

1.79757 —0.00021 

0.00018 3.99173 
— 1.79838 —0.11343. 


Now we can form the matrix product X,X}; it is: 
0.79568 —0.00021 3.99173 —0.11343 
1.43029 —0.00038 7.17541 —0.20390 


0.00014 0 0.00072 —0.00002 
— 1.43093 0.00038 —7.17865 0.20399 


X,X4 = 


Next we form the matrix A: 
Ay = A-AX,X} 
0.20432 0.00021 —2.99173 0.11343 
— 0.43029 0.77816 —6.84208 0.53723 


—0.00014 —-0.02525 0.55484 —0.02523 
1.43093 —0.88927 -—1.46579 —0.09288 


and form the iterates of the vector Yy=(1, 1, 1, 1) by this matrix. 
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We adduce the 17th and 18th iterates of the matrix A): 


AVY, ANY, 
—0.00869 —0.00580 
—0.05388 —0.03597 

0.00290 0.00193 

0.04107 0.02741 


—0.01861 -—0.01242. 


The ratios of the components of the 17th and 18th iterates give 
for Ay the values 


0.667 
0.668 
0.666 
0.667 . 


The ratios of the corresponding components of the 18th iterate 
give for the components of the second proper vector the values 


1.00 
6.20 
— 0.333 
—4.73. 


We sec that not only the second proper number A, but also the 
components of the second proper vector are determined more ex- 
actly by the method of exhaustion than by the 4-difference method. 
However this method requires, for the case of a nonsymmetric 
matrix, much additional work. In the case of a symmetric matrix 
the method of exhaustion can be recommended. 

In computing the second proper number and the components of 
the second proper vector, one can utilize the modification of the 
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process that was described, whereby the vector AfY, is computed, 
but the iterations of the matrix A, are avoided, by the formula 


ALY, = A'Y,—ARX,X(Yp. 
Making the computation, we obtain 


4.67377 
8.40142 
0.00084 
— 8.40521 


A1X1¥ = 


Now we compute AtY, for k=9 and 10. This gives 


k=9 k=10 
—0.20724 -—0.13725 
—1.27708 -0.86065 
0.06915 0.04679 
0.93446 0.63739 . 
Hence we find Ag: 
0.662 
0.674 
0.677 
0.682 


and from A}°Yo, the components of the proper vector: (1.00, 6.27, 
—0.34, — 4.64). 

3. The reduction method. Given the matrix A, let the first proper 
number 4, and the proper vector XY, =(x,,...,%,) belonging to it 
have been computed. Let us consider the matrix 


xy 0... 0 
P= xX | 0 
x, O 1 
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We confirm without difficulty that 


— oO 0 
*1 
Xe 

Pic ard ] 0 
—* 9 1 
*) 


The matrix P-1AP is similar to the matrix A, and the proper num- 
bers of both matrices are thus identical. 


But 
212 a 
A —_— — 
1 xy 
x x 
: 0 ao.—-—a,9 ... @y—-— a 
P-\AP = 22 a2 aay, Ain 
0 a e—— aye Ang —— Fin 
A, dis bin 
0 
: B 
0 
Thus 


|P-1AP—Al| = (a,—A)|B—Al]. 


Consequently the proper numbers of the matrix A will be A, and 
the proper numbers of B, a matrix of the (n—1)th order. If X; be 
so normalized that x;=1, we will have 


A@gag—XeGi9 +++ Baqn—X2F1_ 


Ang —%_p2@190 + + + Ong — XA AI y 
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For the determination of Ag we must obviously construct a sequence 
BYo,..., BkYo,... and find A, as the ratio of any components of 
the constructed vectors. 

Furthermore let Y be a proper vector of the matrix B. The 


matrix P-'AP will then have a proper vector (*). Let us deter- 


mine y}. 
We have 


Ay 430 «2+ Gy 
0 ¢ ) 
B Y 


= (aiamae 18 oe) - (4 ) 
BY y/}° 


Equating the first components of this vector equation, we obtain 


Ayyitayyot +++ +4, = Ay, 
whence 
ie 4 oYot 22+ $4 1nYn | 
t A-A, 


Finally the proper vector X; of the matrix A is determined by the 
formula 
(i) 


yy 
xn r(2) = [ oP? 
: Yep eh fh OR 
na 44? 


Example. Let us determine the second proper number and the 
components of the proper vector belonging to it, for the matrix of 
§ 30, Paragraph 2. 

As we have seen, the first proper number for this matrix has been 
determined to be 4,~1.001 (§ 30, Paragraph 2),.and the com- 
ponents of the proper vector belonging to it to be (1, 1.79757, 
0.00018, — 1.79838) (§ 33). 
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For determining A. we compute the matrix B: 
0.77778 —1.46424 0.33333 
B= {| -—0.02525 0.55538 —0.02525 
—0.88889 —6.84606 OLLI 


Next we form iterates of the vector Yy=(1, 1, 1) by the matrix B. 
We give the result of the 15th and 16th iterations: 


BiSX, BX, 
—0.09565 -—0.06388 

0.00725 0.00484 

0.06339 0.04243 


—0.02502 —0.01661. 


We determine A, from the ratios of the components of the 16th 
and 15th iterates. This gives 
0.668 
0.668 
0.669 . 


We now determine the components of the second proper vector. 
Wich this in view, we determine, to begin with, 


= Meet arava + araye _ 0.00484 | _ 9 145, 





y a “20853 - 
Next we compute the components x2, x3, x; of vector Xo: 
X_ = —0.0900 
x3 = 0.00484 
x, = 0.0685. 


Thus the components of the vector X, are (—0.0145, —0.0900, 
0.00484, 0.0685), or, after normalization, (1, 6.21, —0.333, —4.72). 

As we sec, the reduction method makes possible the exact deter- 
mination not only of the second proper number itself but also of the 
proper vector belonging to it. 
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§ 33. DETERMINATION OF THE FIRST 
PROPER NUMBER. SECOND CASE 

Let us now consider the case where the elementary divisors of 
the matrix are linear, but the first proper number 4, is complex. 
In this case A, will also be a proper number, so that of the proper 
numbers that are of greatest modulus there will be not less than two. 
Let us assume that |A3|<|4,|. Let X, and X2 be the proper vectors 
belonging to the proper numbers 4, and A,. On the strength of 
§ 3, Paragraph 6, one may consider their components to be complex 
conjugates. Furthermore, if 


Yo = 4,X;+a9X_+ mee +a,X,, 


is a real initial vector, a, and ag will be complex conjugate numbers. 
Reasoning just as in § 29, we form the sequence of vectors 


(1) AYo,..., A*¥o,... 


Any component of the vector Y,=<A*Yq will obviously have its 
previous form, 


(2) Ye = AL Hest +++ + egAns 
where ¢, and ¢, are complex conjugates. Let 


C) = Reta, co = Re-ie, 


(3) ; ms 

Ay = rei®, Ag = remt8; 
then 
(4) % = 2Rr* cos (kKO+a) +egdh+ - 


The presence of the factor cos(k@+<«) will be the cause of a marked 
oscillation of the values of y,, in magnitude and in sign as well. Thus 
the presence of complex roots that are the largest in modulus will at 
once be revealed in forming the iterates. Since r2=A,A,, we can 
determine the quantity r2 by means of the determinants introduced 
in §31. Here, as distinct from the case of real roots, we will not 
encounter the disappearance of significant figures, so that r? is deter- 
mined with sufficient accuracy from the ratio of these determinants. 
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We shall show that having computed 7, one can determine cos 0, 
utilizing the sequence 4), Yo, -- +5 Y4,+-- for this purpose. 
We introduce the expression! 


(5) Me = Bryer yg). 
A 
We have from (4), accurately to terms of the order of (3) , that 
1 
y, ~ 2Rrt cos(ké +a). 


Furthermore, since 
cos((k+1)0+a] +cos[(A—1)0+a] = 2 cos(k@ +a) cos 0, 


we obviously have 


6 cos 0 ~ HE, 
(6) a 


3 ae A3\t 
with an accuracy to quantities of the order of (2) : 
1 


Since the case of complex roots is of little practical interest, we 
shall not dwell on the determination of the proper vector. For the 
same reason an illustrative example is not given. We have touched 
upon this case as upon the cases considered in the preceding section, 
only in order to show the possible peculiarities of the course of the 
iterative process. 


§34. THE CASE OF A MATRIX WITH NON- 
LINEAR ELEMENTARY DIVISORS 


In all the preceding sections devoted to iterative methods, it has 
been assumed that the elementary divisors of the matrix are linear, 
ie., that the matrix A is reducible to diagonal form. In this section 
we abandon that requirement. As has already been noted in the 
introduction, the iterative process depends essentially on the struc- 
ture of the Jordan canonical form that is connected with the given 
matrix. We shall not consider the general case, but shall show in a 
most simple example only the character of the changes that occur if 
the matrix has nonlinear elementary divisors. 

1 A. Aitken [1]. 
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Let us concern ourselves with the determination of the first proper 
number A, of the matrix A. We shall discuss the case in which 2, 


is real and belongs to the box ( i: ) in the Jordan canonical 
1 


form, and the next proper number A, is less than 4, in modulus. To 
simplify the calculation we shall consider the elementary divisors 
that correspond to the rest of the proper numbers to be linear. 

As previously, we adopt an arbitrary vector Yo and form the 
sequence AYp, A?Yo,..., 44 ¥o,... . We shall elucidate how any 
component y, of the vector Y, = AY depends on the corresponding 
powers of A; in such a case. Since the proper vectors of the matrix 
do not in this case form a basis, let us take as X,,..., X,, the vectors 
of that basis relative to which the linear transformation with the 
matrix A is brought into canonical form. Then 


AX, = AX + Xs 


AX, = A Xe 
(1) AX; = A3X 
AX, = AX, 
Let 
(2) xX; = (x1; Kayy ees Xqj)s Yu = (Yias- ++ 5 Yat)» 
and 


Yo = aX, +aXo+ eee +4,X,. 
We have, on the strength of (1): 
(4) AVg = aX +0, X q+ agd,Xq + agdgX y+ +++ +4,4,%, 


(5) AY, = a,AtX, +aykat Xe + adh Xo +agdgXq+ eee +a,AbX,. 
Furthermore, in view of (2), we have 


Yu = ay Aix + (aay tk +agAt)x;0 + 4g]5%;3+ acts +4,A8%;, 
= (04x;1 + do%;9)A} +a ,X;9kAy' + agxigAgt +++ +4gXjqde- 
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Thus any component of the vector Y, will have the form (we omit 
the first index, as before) : 


(6) yy = CAR tegkAy!+egdht --- +0,A5. 


The ratio 


ra tends to 4, as before, but more slowly than any 
k 


geometrical progression, owing to the presence of the factor & in the 


second term, to wit: 
Yete yy, 
LT; . af! +0(3)| 


To determine A, from the ratio 2+! becomes practically im- 
1 Ws Pp y 


possible. 

We note that if the box to which A, belongs in the Jordan canonical 
form has a more complex structure, other powers of A, also appear 
in expression (6); they are multiplied by the corresponding binomial 
coefficients: 


k(k—1) qe? 


—o » +¢,a4. 


yy = CA + eakah” 14, 


" 


The ratio 4*! approaches A, still more slowly. 


In ey a slow convergence of the process provides grounds for 
surmising that a nonlinear elementary divisor, connected with the 
largest proper number, is present. 

However the process by means of which the product of two proper 
numbers is determined will converge rapidly enough in the case in 
hand. It can be confirmed without difficulty that 


2 A 
7 Cyita Yero) _ ai + 0 2) 
” [Yar Yeti] : A 
for indeed on computing [y,, y:41) = Yk Yet! we find that 
Yeti Vere 








k 
[Ye Yer] = —GARA + o(?) , whence (7) follows. 


Ratio (7) makes possible the determination of A). 
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We shall now show how, in case A, belongs to the box ( ? : ) 
1 


and this circumstance is suspected from the slow convergence of the 
sequence ¥,, one can satisfy oneself that this is indeed the fact. 
Let us have found 4, from the ratio (7). Let us construct 


Ay, = yeri— Ary, = Cay"! + 00(k+1)Ai teat + + - 





(8) +6,Aht} —c,4;*! —cokAj — CAA; ba tice, —¢, AA 
= 2A} + O(A5). 
Then 
Ays+y = As\* 
Ay, - 1,+0(2) : 
i.e., aye tends to A, fast enough. The coincidence of the limit of 
k 
es with the value for 2, computed earlier, and the fact of the 
k 


rapid convergence of Ayes 





to A, serve as confirmation of the surmise 


that A, figures in a box of the second order. 

Instead of the computation described, one can have recourse to 
computation of the second /-difference. 

If A, belongs to a box of the second order, the second A-difference 


Aty, = Ayi1— Ady, = O(A3) 


will be small in comparison with the component y, itself. 

The case of nonlinear elementary divisors of the matrix is very 
rarely met with in practice. We shall therefore not dwell either on 
the determination of the proper vector that belongs to the first 
proper number, or on the determination of the succeeding proper 
numbers and the vectors belonging to them. 

We remark only that all the devices we have discussed for the case 
of linear elementary divisors may be employed in the general case 
too, after suitable modification. ! 


1A. Aitken [1]; K. A. Semendiaev [1]. 
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§35. IMPROVING THE CONVERGENCE OF 
THE ITERATIVE PROCESS FOR SOLVING 
SYSTEMS OF LINEAR EQUATIONS! 


As we saw in § 17, in solving a system of linear equations given 
in the form X = AX+F by the method of iteration, we consider that 
XxX) =AX4*-+F, Here 


(1) X—X) = (XN — XW) + (XD — XUD) He, 


Let us calculate approximately the sum of this series. 
We have 


(2) Xk XW) = A(X XUV) see. = AMX — XO), 


Let 4,,..., 4, be the proper numbers of the matrix A; we shall 
consider them to be distinct, for simplicity, and to be arranged in 
order of diminishing moduli, and shall consider U,, Us,..., U, to 
be the proper vectors that correspond to them. Let us resolve the 
vector X() — X (© in terms of the proper vectors of the matrix A: 


X') — XO) = a,U,+ eae +a,U,. 


Then, on the strength of (2): 
Pekin ds = a AU + os +a,A\U, = 0, 4,U, + O(a). 


As was shown in § 29, we can determine the first proper number 
A, from the ratio of the components of the vectors (X +) — X)) and 
(X() — X4-D), with sufficiently large &. Moreover 


(X4+D — Xb) 4 (NUH) — XU) 4 
(3) = a Ai(Lt Ay Fait «+ -)U + +) Fa, AKL 4+ at 22+ ++ -)U, 


aah a,A* > XetD_ y® 
=a eee ia 








eet O(A2). 

We have the right to sum the geometrical progressions, for all 

|A;] <1, since we consider the iterative process to be convergent. 
tL. A. Liusternik (1). 
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Thus, accurate to quantities of the order of 4k, we have, as the solu- 
tion of the linear system, the expression 


(k+l) — Xk) 
(4) X~ XH4 Sik ek Col 
I-A, 
The extra term in equation (4) is effortlessly computed, and greatly 
improves the convergence of the iterative process. 
If the solution of the linear system is approximately determined by 


k 
‘means of the expression X~ > A/F, then, reasoning analogously, 
i=l 





we find that 
k kel 
va > app 
isl 1-A, 


will give a better approximation to the sought solution. Here the 
quantity 4, is found approximately from the ratio of the com- 
ponents of the vectors A‘t1F and A‘F. 
Thus for the example analysed by us in §17, we obtained (see 
§ 29), on proceeding from the 14th approximation, 4, =0.4800. 
Furthermore, from Table XIII, § 17, we have 


X43) = (1.53490847, 0.12200958, 1.97509985, 1.41289889). 








atl 
On computing 4 7, we obtain 
A 
-1 
« ra = (0.00005656, 0.00005656, 0.00005656). 
—Ay 


Thus 
X ~ (1.534965, 0.122010, 1.975156, 1.412955). 


We see that, with accuracy to the sixth figure, the improved solution 


coincides with that found by Gauss’s method. 
Moreover, 


6 
> AW = (1.52533, 0.12201, 1.96551, 1.40333). 
i=0 
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Having taken A, =0.4792 (see § 29) and used the device for im- 
provement we find that 


6 
> AIF+ oe 


0.5008 = (1.53495, 0.12203, 1.97514, 1.41293). 
i= 


As we see from Table XIII of § 17, the approximation obtained 
coincides better with the exact solution than does the 14th approxi- 
mation. 

The method for improving the convergence here suggested may 
also be applied to the Seidel method, for that method, applied to 
the system X=AX+F, is equivalent to application of the usual 
iterative method to the system 


X = A,X+F, where -l, = (J-B)-1C. 


Here B and C are the triangular constituent matrices of the 
matrix A (see § 19). The largest proper number #2, of the matrix 
A, is determined from the ratio of the components of the vectors 
X k+l) — Xe) and X#) — Xk-)), and for improving the convergence 
one can as before utilize the formula 


Xke+y — Xt) 


Xw XH4 = 
ae a | 


In conclusion we make mention of methods that have quite 
recently appeared in the literature, which are based on the ideas of 
functional analysis. These methods, developed for the solution of 
more general problems of operational calculus, give, in particular, 
a solution of the basic problems of linear algebra (not only the 
solution of a linear algebraic system, but also the finding of the 
proper numbers of a matrix), if for the operator under study one 
adopts the linear transformation connected with the given matrix. 

The reader who is interested in these methods and who is familiar 
with the basic ideas of functional analysis is referred to the extensive 
article of L. V. Kantorovich [1], which contains a large biblio- 
graphy, and also to another work of the same author, [2], as well 
as to those of I. P. Natanson [1], M. Sh. Birman [1], and M. K. 
Gavurin [1]. 
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THe NoTepooxs OF LEONARDO DA ViNCl, edited by J.P. Richter. Extracts from 
manuscripts reveal great genius; on painting, sculpture, anatomy, sciences, geo- 
graphy, etc. Both Italian and English. 186 ms. pages reproduced, plus 500 addi- 
tional drawings, including studies for Last Supper, Sforza monument, etc. 860pp. 
77 x 10%. USO 22572-0, 22573-9 Pa., Two vol. set $12.00 


Ant Nouveau Desicxs in Cotor, Alphonse Mucha, Maurice Verneuil, Georges 
Auriol. Full-color reproduction of Combinaisons ornamentales (c. 1900) by Art 
Nouveau masters. Floral, animal, geometric, interlacings, swashes — borders, 
frames, spots —all incredibly beautiful. 60 plates, hundreds of designs. 998 x 
8lh6 . 22885-1 Pa. $4.00 


Grapuic Wonks oF OpiLon Repon. All great fantastic lithographs, etchings, 
engravings, drawings, 209 in all. Monsters, Huysmans, still life work, etc. Intro- 
duction by Alfred Werner. 209pp. 91/5 x 124. 21996-8 Pa. $6.00 


Exotic Flora Pattenns In Coor, E.-A. Seguy. Incredibly beautiful full-color 
pochoir work by great French designer of 20's. Complete Bouquets et fron- 
daisons, Suggestions pour étoffes. Richness must be seen to be believed. 40 plates 
containing 120 patterns. 80pp. 95/8 x 12!4. 23041-4 Pa. $6.00 


SELECTED Ercuincs oF James A. McN. WuiIsTLER, James A. McN. Whistler. 149 
outstanding etchings by the great American artist, including selections from the 
Thames set and two Venice sets, the complete French set, and many individual 
prints. Introduction and explanatory note on each print by Maria Naylor. 157pp. 
93/8 x 1214. 23194-1 Pu. $5.00 


Visuat ILLusions: THEIR Causes, CHARACTERISTICS, AND APPLICATIONS, Matthew 
Luckiesh. Thorough description, discussion; shape and size, color, motion; 
natural illusion. Uses in art and industry. 100 illustrations. 252pp. 

21530-X Pa. $2.50 


TEN Books ON ARCHITECTURE, Vitruvius. The most important book ever written on 
architecture. Early Roman aesthetics, technology, classical orders, site selection, 
all other aspects. Stands behind everything since. Morgam translation. 331pp. 

20645-9 Pa. $3.50 


Tut Copex Netrane. A Picture Masvuscaipt From ANCIENT MEXICO, as first 
edited by Zelia Nuttall. Only inexpensive edition, in full color, of a pre-Colum- 
bian Mexican (Mixtec) book. 88 color plates show kings, gods, heroes, temples, 
sacrifices. New explanatory, historical introduction by Arthur G. Miller. 96pp. 
115 x 814. 23168-2 Pa. $7.50 
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AGAINST THE GRAIN (A Resours), Joris K. Huysmans. Filled with weird images, 
evidences of a bizarre imagination, exotic experiments with hallucinatory drugs, 
rich tastes and smells and the diversions of its sybarite hero Duc Jean des Esscintes, 
this classic novel pushed 19th-century literary decadence to its limits. Full un- 
abridged edition. Do not confuse this with abridged editions gencrally sold. Intro- 
duction by Havelock Ellis. xlix -+- 206pp. 22190-3 Paperbound $2.50 


VARIORUM SHAKESPEARE: HAMLET. Edited by Horace H. Furness; a landmark 
of American scholarship. Exhaustive footnotes and appendices treat all doubtful 
words and phrases, as well as suggested critical emendations throughout the play's 
history. First volume contains editor's own text, collated with all Quartos and 
Folios. Second volume contains full first Quarto, translations of Shakespeare's 
sources (Belleforest, and Saxo Grammaticus), Der Bestrafte Brudermord, and many 
essays on critical and historical points of interest by major authorities of past and 
present. Includes details of staging and costuming over the years. By far the 
best edition available for serious students of Shakespeare. Total of xx -+ 905pp. 

21004-9, 21005-7, 2 volumes, Paperbound $11.00 


A LIFE OF WILLIAM SHAKESPEARE, Sir Sidney Lee. This is the standard life of 
Shakespeare, summarizing everything known about Shakespeare and his plays. 
Incredibly rich in material, broad in coverage, clear and judicious, it has served 
thousands as the best introduction to Shakespeare. 1931 edition. 9 plates. 
xxix -+ 792pp. 21967-4 Paperbound $4.50 


MASTERS OF THE DRAMA, John Gassner. Most comprchensive history of the drama 
in print, covering every tradition from Greeks to modern Europe and America, 
including India, Far East, etc. Covers more than 800 dramatists, 2000 plays, with 
biographical material, plot summaries, theatre history, criticism, etc. “Best of its 
kind in English,” New Republic. 77 illustrations. xxii 4+- 890pp. 

20100-7 Clothbound $10.00 


THE EVOLUTION OF THE ENGLISH LANGUAGE, George McKnight. The growth 
of English, from the 14th century to the present. Unusual, non-technical account 
presents basic information in very interesting form: sound shifts, change in grammar 
and syntax, vocabulary growth, similar topics. Abundantly illustrated with quota- 
tions. Formerly Modern English in the Making. xii +- 590pp. 

21932-1 Paperbound $4.00 


AN EtyMoLoGicaL DicTIONARY OF MopERN ENGuISiHi, Ernest Weekley. Fullest, 
richest work of its sort, by foremost British lexicographer. Detailed word histories, 
including many colloquial and archaic words; extensive quotations. Do not con- 
fuse this with the Concise Etymological Dictionary, which is much abridged. Total 
of xxvii + 830pp. 64, x 9%. 

21873-2, 21874-0 Two volumes, Paperbound $10.00 


FLATLAND: A ROMANCE OF MANY DIMENSIONS, E. A. Abbott. Classic of 
science-fiction explores ramifications of life in a two-dimensional world, and what 
happens when a three-dimensional being intrudes. Amusing reading, but also use- 
ful as introduction to thought about hyperspace. Introduction by Banesh Hoffmann. 
16 illustrations. xx -+- 103pp. 20001-9 Paperbound $1.50 
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AusTRIAN COOKING AND BakinG, Gretel Beer. Authentic thick soups, wiener 
schnitzel, veal goulash, more, plus dumplings, puff pastries, nut cakes, sacher 
tortes, other great Austrian desserts. 224pp. USO 23220-4 Pa. $2.50 


CHEESES OF THE WorRLD, U.S.D.A. Dictionary of cheeses containing descriptions of 
over 400 varieties of cheese from common Cheddar to exotic Surati. Up to two 
pages are given to important cheeses like Camembert, Cottage, Edam, etc. 
15Ipp. 22831-2 Pa. $1.50 


Tritton’s GUIDE TO BETTER WINE AND BEER MAKING FOR BEGINNERS, S.M. Tritton. 
All you need to know to make family-sized quantities of over 100 types of grape, 
fruit, herb, vegetable wines; plus beers, mead, cider, more. 11 illustrations. 
157pp. USO 22528-3 Pa, $2.25 


DECORATIVE LABELS FOR HOME CANNING, PRESERVING, AND OTHER HOUSEHOLD AND 
Girt Uses, Theodore Menten. 128 gummed, perforated labels, beautifully 
printed in 2 colors. 12 versions in traditional, Art Nouveau, Art Deco styles. 
Adhere to metal, glass, wood, most plastics. 24pp. 8'4 x 11. 23219-0 Pa. $2.00 


Five ACRES AND INDEPENDENCE, Maurice G. Kains. Great back-to-the-land classic 
explains basics of self-sufficient farming: economics, plants, crops, animals, 
orchards, soils, land selection, host of other necessary things. Do not confuse with 
skimpy faddist literature; Kains was one of America’s greatest agriculturalists. 95 
illustrations. 397pp. 20974-1 Pa. $3.00 


GrowinG VEGETABLES IN THE HoME GarbEN, U.S. Dept. of Agriculture. Basic in- 
formation on site, soil conditions, selection of vegetables, planting, cultivation, 
gathering. Up-to-date, concise, authoritative. Covers 60 vegetables. 30 illustra- 
tions. 123pp. 23167-4 Pa. $1.35 


FRUITS FOR THE HOME GanveN, Dr. U.P. Hedrick. A chapter covering each type of 
garden fruit, advice on plant care, soils, grafting, pruning, sprays, transplanting, 
and much more! Very full. 53 illustrations. 175pp. 22944-0 Pa. $2.50 


GARDENING ON SANDY SOIL IN NORTH TEMPERATE AREAS, Christine Kelway. Is your 
soil too light, too sandy? Improve your soil, select plants that survive under such 
conditions. Both vegetables and flowers. 42 photos. 148pp. 

USO 23199-2 Pa. $2.50 


THE FRAGRANT GARDEN: A Book ABOUT SWEET SCENTED FLOWERS AND LEAVES, 
Louise Beebe Wilder. Fullest, best book on growing plants for their fragrances. 
Descriptions of hundreds of plants, both well-known and overlooked. 407pp. 

23071-6 Pa. 34.00 


Easy GARDENING WITH DROUGHT-RESISTANT PLANTS, Arno and Irene Nehrling. 
Authoritative guide to gardening with plants that require a minimum of water: 
seashore, desert, and rock gardens; house plants; annuals and perennials; much 
more. 190 illustrations. 320pp. 23230-1 Pa. $3.50 
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DECORATIVE ALPHABETS AND INITIALS, edited by Alexander Nesbitt. 91 complete 
alphabets (medieval to moder), 3924 decorative initials, including Victorian 
novelty and Art Nouveau. 192pp. 7% x 10%. 20544-4 Pa. $4.00 


Catuicnapny, Arthur Baker. Over 100 original alphabets from the hand of our 
greatest living calligrapher: simple, bold, fine-line, richly ornamented, etc. —all 
strikingly original and different, a fusion of many influences and styles. 155pp. 
1138 x 8A. 22895-9 Pa. $4.50 


MONOGRAMS AND ALPHABETIC Devices, edited by Hayward and Blanche Cirker. 
Over 2500 combinations, names, crests in very varied styles: script engraving, or- 
nate Victorian, simple Roman, and many others. 226pp. 8's x 11. 

22330-2 Pa. $5.00 


Tue Book oF Sicns, Rudolf Koch. Famed German type designer renders 493 sym- 
hols: religious, alchemical, imperial, runes, property marks, etc. Timeless. 104pp. 
6l/8 x 9. 20162-7 Pa. $1.75 


200 Decorative Tit. Paces, edited by Alexander Nesbitt. 1478 to late 1920's. 
Baskerville, Diirer, Beardsley, W. Morris, Pyle, many others in most varied tech- 
niques. For posters, programs, other uses. 222pp. 834 x 11'4. 21264-5 Pa. $5.00 


DicTIONARY OF AMENICAN Portraits, edited by Hayward and Blanche Cirker. 
4000 important Americans, earliest times to 1905, mostly in clear line. Politicians, 
writers, soldiers, scientists, inventors, industrialists, Indians, Blacks, women, out- 
laws, etc. Identificatory information. 756pp. 9'4 x 12%. 21823-6 Clothhd. $30.00 


Ant Forms in Nature, Ernst Haeckel. Multitude of strangely beautiful natural 
forms: Radiolaria, Foraminifera, jellyfishes, fungi, turtles, bats, etc. All 100 plates 
of the 19th century evolutionist’s Kunstformen der Natur (1904). 100pp. 99/5 x 
12%. 22987-4 Pa. $4.00 


DecourpacE: Tue BiG Picture Sourcesook, Eleanor Rawlings. Make hundreds of 
beautiful objects, over 550 florals, animals, letters, shells, period costumes, 
frames, etc. selected by foremost practitioner. Printed on one side of page. 8 col- 
or plates. Instructions. 176pp. 94/6 x 12%. 23182-8 Pa. $5.00 


AMERICAN FoLkK DecoraTios, Jean Lipman, Eve Meulendyke. Thorough coverage 
of all aspects of wood, tin, leather, paper, cloth decoration — scapes, humans, 
trees, flowers, geometrics — and how to make them. Full instructions. 233 il- 
lustrations, 5 in color. 163pp. 83 x 114. 22217-9 Pa. $3.95 


WUHITTLING AND WooncaavinG, E.J. Tangerman, Best book on market; clear, full. 
If you can cut a potato, you can carve toys, puzzles, chains, caricatures, masks, 
patterns, frames, decorate surfaces, etc. Also covers serious wood sculpture. Over 
200 photos. 293pp. 20965-2 Pa. $3.00 
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East O' THE SUN AND WEST O' THE Moon, George W. Dasent. Considered the 
best of all translations of these Norwegian folk tales, this collection has been enjoyed 
by generations of children (and folklorists too). Includes True and Untrue, Why the 
Sea is Salt, East O° the Sun and West O' the Moon, Why the Bear is Stumpy-Tailed, 
Boots and the Troll, The Cock and the Hen, Rich Peter the Pedlar, and 52 more. 
The only edition with all 59 tales. 77 illustrations by Erik Werenskiold and Theodor 
Kittelsen. xv ++ 418pp. 22521-6 Paperbound $4.00 


Goops ANp How To BB THEM, Gelett Burgess. Classic of tongue-in-cheek humor, 
masquerading as etiquette book. 87 verses, twice as many cartoons, show mis- 
chievous Goops as they demonstrate to children virtues of table manners, neatness, 
courtesy, etc. Favorite for generations. viii -+- 88pp. 6/2 x 914. 

22233-0 Paperbound $2.00 


ALICE'S ADVENTURES UNDER GROUND, Lewis Carroll. The first version, quite 
different from the final Alice in Wonderland, printed out by Carroll himself with 
his own illustrations. Complete facsimile of the “million dollar” manuscript Carroll 
gave to Alice Liddell in 1864. Introduction by Martin Gardner. viii + 96pp. Title 
and dedication pages in color. 21482-6 Paperbound $1.50 


Tue BROWNIES, THEIR BOOK, Palmer Cox. Small as mice, cunning as foxes, exu- 
berant and full of mischief, the Brownies go to the zoo, toy shop, seashore, circus, 
etc., in 24 verse adventures and 266 illustrations. Long a favorite, since their first 
appearance in St. Nicholas Magazine. xi-+ 144pp. 64 x 914. 

21265-3 Paperbound $2.50 


SONGS OF CHILDHOOD, Walter De La Mare. Published (under the pseudonym 
Walter Ramal) when De La Mare was only 29, this charming collection has long 
been a favorite children’s book. A facsimile of the first edition in paper, the 47 poems 
capture the simplicity of the nursery rhyme and the ballad, including such lyrics as 
I Met Eve, Tartary, The Silver Penny. vii +- 106pp. (USO) 21972-0 Paperbound 

$2.00 


THE CoMPLETE NONSENSE OF Epwarb LEAR, Edward Lear. The finest 19th-century 
humorist-cartoonist in full: all nonsense limericks, zany alphabets, Owl and Pussy- 
cat, songs, nonsense botany, and more than 500 illustrations by Lear himself. Edited 
by Holbrook Jackson. xxix -+ 287pp. (USO) 20167-8 Paperbound $3.00 


BILLY WHISKERS: THE AUTOBIOGRAPHY OF A GOAT, Frances Trego Montgomery. 
A favorite of children since the early 20th century, here are the escapades of that 
rambunctious, irresistible and mischievous goat—Billy Whiskers. Much in the 
spirit of Peck's Bad Boy, this is a book that children never tire of reading or hearing. 
All the original familiar illustrations by W. H. Fry are included: 6 color plates, 
18 black and white drawings. 159pp. 22345-0 Paperbound $2.75 


MoTHER Goose Me opis. Faithful republication of the fabulously rare Munroe 
and Francis ‘copyright 1833" Boston edition—the most important Mother Goose 
collection, usually referred to as the “original.” Familiar rhymes plus many rare 
ones, with wonderful old woodcut illustrations. Edited by E. F. Bleiler. 128pp. 
Ay x 63%. 22577-1 Paperbound 31.50 


CATALOGUE OF DOVER BOOKS 


Hovpin1 ox Macic, Harald Houdini. Edited hy Walter Gibson, Morris N. Young. 
How he escaped; exposés of fake spiritualists; instructions for eye-catching 
tricks; other fascinating material by and about greatest magician. 155 illustra- 
tions. 280pp. 20384-0 Pa. $2.75 


HANDBOOK OF THE NUTRITIONAL CONTENTS OF Foop, U.S. Dept. of Agriculture. 
Largest, most detailed source of food nutrition information ever prepared. Two 
mammoth tables: one measuring nutrients in 100 grams of edible portion; the 
other, in edible portion of 1 pound as purchased. Originally titled Composition of 
Foods. 190pp. 9 x 12. 21342-0 Pa. $4.00 


CoMPLETE GUIDE TO HOME CANNING, PRESERVING AND FREEZING, U.S. Dept. of 
Agriculture. Seven basic manuals with full instructions for jams and jellies; 
pickles and relishes; canning fruits, vegetables, meat; freezing anything. Really 
good recipes, exact instructions for optimal results. Save a fortune in food. 156 il- 
lustrations. 2)4pp. 64 x 914. 2291 1-4 Pa. $2.50 


Tue: BREAD Tray, Louis P. De Gouy. Nearly every bread the cook could buy or 
make: bread sticks of Haly, fruit breads of Greece, glazed rolls of Vienna, every- 
thing from corn pone to croissants. Over 500 recipes altogether. including buns, 
rolls, muffins, scones, and more. 463pp. 23000-7 Pa. $3.50 


Creative Hampurcer Cookery, Louis P. De Gouy. 182 unusual recipes for 
casseroles, meat loaves and hamburgers that turn inexpensive ground meat into 
memorable main dishes: Arizona chili burgers, burger tamale pie, burger stew, 
burger corn loaf, burger wine loaf, and more. 120pp. 23001-5 Pa. $1.75 


Lone Isuaxp SEAFOOD Cooxnook, J. George Frederick and Jean Joyce. Probably 
the best American seafood cookbook. Hundreds of recipes. 40 gourmet sauces, 
123 recipes using oysters alone! All varieties of fish and seafood amply repre- 
sented. 324pp. 22677-8 Pa. $3.50 


Tue Epicuncan: A COMPLETE TREATISE OF ANALYTICAL AND PRACTICAL STUDIES IN 
‘tHe Cuninary Art, Charles Ranhofer. Great modern classic. 3,500 recipes from 
master chef of Delmonico’s, turn-of-the-century America’s best restaurant. Also 
explained, many techniques known only to professional chefs. 775 illustrations. 
1183pp. 654 x 10. 22680-8 Clothbd. $22.50 


Tne AMERICAN WINE Cook Book, Ted Hatch. Over 700 recipes: old favorites 
livened up with wine plus many more: Czech fish soup, quince soup, sauce 
Perigueux, shrimp shortcake, filets Stroganoff, cordon bleu goulash, jambonneau, 
wine fruit cake, more. 31 4pp. 22796-0 Pa. $2.50 


DeELicious VEGETARIAN Cookinc, Ivan Baker. Close to 500 delicious and varied 
recipes: soups, main course dishes (pea, bean, lentil, cheese, vegetable, pasta, and 
egg dishes), savories, stews, whole-wheat breads and cakes. more. 168pp. 

USO 22834-7 Pa. $1.75 
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How To SOLVE Cress Prosems, Kenneth §. Howard. Practical suggestions on 
problem solving for very beginners. 58 two-move problems, 46 3-movers, 8 4- 
movers for practice, plus hints. 171 pp. 20748-X Pit. $2.00 


A Guine to Fairy Cirss, Anthony Dickins. 3-D chess, 4-D chess, chess on a 
cylindrical board, reflecting pieces that bounce off edges. cooperative chess, 
retrograde chess, maximummers, much more. Most based on work of great 
Dawson. Full handbook, 100 problems. 66pp. 77 x 10%. 22687-5 Pa. $2.00 


Wis at BackGaMMos, Millard Hopper. Best opening moves, running game. block- 
ing game, back game, tables of odds, etc. Hopper makes the game clear enough 
for anyone to play, and win. 43 diagrams. 11 I pp. 22894-0 Pa. $1.50 


Bippinc A Brice Hasxp, Terence Reese. Master player “thinks out loud” the 
binding of 75 hands that defy point count: systems. Organized by bidding 
problem—no-fit situations, overbidding, underbidding, cueing your defense, ete. 
254pp. EBE 22830-4 Pa. $3.00 


Tie PRECISION BIDDING SySTEM IN BatpGE, C.C. Wei, edited by Alan Truscott. In- 
ventor of precision bidding presents average hands and hands from actual play, 
including yames from 1969 Bermuda Bowl where system emerged. 114 exercises. 
116pp. 21171-1 Pa. $1.75 


Leann Macic, Henry Hay. 20 simple, easy-to-follow lessons on magic for the new 
magician: illusions, card tricks, silks, sleights of hand, coin’ manipulations, 
escapes, and more —all with a minimum amount of equipment. Final chapter ex- 
plains the great stage illusions. 92 illustrations. 285pp. 21238-6 Pa. $2.95 


Tuk New Macictan’s Manvac, Walter B. Gibson. Step-by-step instructions and 
clear illustrations guide the novice in mastering 36 tricks; much equipment sup- 
plied on 16 pages of cut-out materials. 36 additional tricks. 64 illustrations. 
159pp. 65 x 10. 23113-5 Pu. $3.00 


PROFESSIONAL MAGIC FOR AMATEURS, Walter B. Gibson. 50 easy, effective tricks 
used by professionals —cards, string, tumblers, handkerchiefs, mental magic, ete. 
63 illustrations. 223pp. 23012-0 Pa. $2.50 


Carp Maniec ations, Jean Hugard. Very rich collection of manipulations; has 
taught thousands of fine magicians tricks that are really workable, eye-catching. 
Easily followed, serious work. Over 200 illustrations. 163pp. 20539-8 Pa. $2.00 


Annott’s ExcyCLOPebta OF Rope Tricks FOR MAGICIANS, Stewart James. Complete 
reference book for amateur and professional magicians containing more than 150 
tricks involving knots, penetrations, cut and restored rope, ete. 510 illustrations. 
Reprint of 3rd edition. 400pp. 23206-9 Pa. $3.50 


Tue Secrets oF Houpint, J.C. Cannell. Classic study of Houdini'’s incredible 
magic, exposing closely-kept professional secrets and revealing, in general terms, 
the whole art of stage magic. 67 illustrations. 279pp. 22913-0 Pa $2.50 
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Jewistt GREETING Carps, Ed Sibbett, Jr. 16 cards to cut and color. Three say 
“Happy Chanukah,” one “Happy New Year,” others have no message, show stars 
of David, Torahs, wine cups, other traditional themes. 16 envelopes. 8% x 11. 

23225-5 Pa. $2.00 


AUBREY BEARDSLEY GREETING Carb Book, Aubrey Beardsley. Edited by Theodore 
Menten. 16 elegant yet inexpensive greeting cards let you combine your own sen- 
timents with subtle Art Nouveau lines. 16 different Aubrey Beardsley designs 
that you can color or not, as you wish. 16 envelopes. 64pp. 8'4 x 11. 

23173-9 Pa. $2.00 


RECREATIONS IN THE THEORY OF Numnens, Albert Beiler. Number theory, an inex- 
haustible source of puzzles, recreations, for beginners and advanced. Divisors, 
perfect numbers. scales of notation, ete. 349pp. 21096-0 Pa. $4.00 


AMUSEMENTS IN MATHEMATICS, Henry E. Dudeney. One of largest puzzle collec- 
tions, based on algebra, arithmetic, permutations, probability, plane figure dissec- 
tion, properties of numbers, by one of world’s foremost puzzlists. Solutions. 450 
illustrations. 258pp. 20473-1 Pa. $3.00 


MaTHEMATICS, MAGIC AND Mystery, Martin Gardner. Puzzle editor for Scientific 
American explains math behind: card tricks, stage mind reading, coin and match 
tricks, counting out games, geometric dissections. Probability, sets, theory of 
numbers, clearly explained. Plus more than 400 tricks, guaranteed to work. 135 
illustrations. |76pp. 20335-2 Pa. $2.00 


BesT MATHEMATICAL PuzzLes OF SAM Loyn, edited by Martin Gardner. Bizarre, 
original, whimsical puzzles by America’s greatest puzzler. From fabulously rare 
Cyclopedia, including famous 14-15 puzzles, the Horse of a Different Color, 115 
more. Elementary math. 150 illustrations. 167pp. 20498-7 Pa. $2.50 


MATHEMATICAL PUZZLES FOR BEGINNERS AND ENTHUSIASTS, Geoffrey Mott-Smith. 
189 puzzles from easy to difficult involving arithmetic, logic, algebra, properties 
of digits, probability. Explanation of math behind puzzles. 135 illustrations. 
248pp. 20198-8 Pa.$2.75 


Bic Book oF Mazes AND Lanvnintus, Walter Shepherd. Classical, solid, and ripple 
mazes; short path and avoidance labyrinths, more —50 mazes and labyrinths in 
al!. 12 other figures. Full solutions. 112pp. 84% x 11. 22951-3 Pa. $2.00 


Coin GaMEs AND Puzzies, Maxey Brooke. 60 puzzles, games and stunts — from 
Japan, Korea, Africa and the ancient world, by Dudency and the other great 
puzzlers, as well as Maxey Brooke’s own creations. Full solutions. 67 illustrations. 
94pp. 22893-2 Pa. $1.50 


Hanp SuaDows TO BE THROWN UPON THE WaLL, Henry Bursill. Wonderful Vic- 
torian novelty tells how to make flying birds, dog, goose, deer, and 14 others. 
32pp. 64 x OIA. 21779-5 Pa. $1.25 
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MANUAL OF THE TREES OF NorTH AMERICA, Charles S. Sargent. The basic survey of 
every native tree and tree-like shrub, 717 species in all. Extremely full descrip- 
tions, information on habitat, growth, locales, economics, etc. Necessary to every 
serious tree lover. Over 100 finding keys. 783 illustrations. Total of 986pp. 

; 20277-1, 20278-X Pa., Two vol. set $9.00 


Binns oF THE New York AREA, John Bull. Indispensable guide to more than 400 
species within a hundred-mile radius of Manhattan. Information on range, status, 
breeding, migration, distribution trends, etc. Foreword by Roger Tory Peterson. 
17 drawings; maps. 540pp. 23222-0 Pa. $6.00 


Tne SEA-BEACH AT Eps-T1pE, Augusta Foote Arnold. Identify hundreds of marine 
plants and animals: algae, seaweeds, squids, crabs, corals, etc. Descriptions cover 
food, life cycle, size, shape, habitat. Over 600 drawings. 490pp. 

21949-6 Pa. $5.00 


Tur Motu Book, William J. Holland. Identify more than 2,000 moths of North 
America. General information, precise species descriptions. 623 illustrations plus 
48 color plates show almost all species, full size. 1968 edition. Still the basic book. 
Total of 551pp. 6% x 94. 21948-8 Pa. $6.00 


AN INTRODUCTION TO THE REPTILES AND AMPHIBIANS OF THE UNITED STATES, Percy 
A. Morris. All lizards, crocodiles, turtles, snakes, toads, frogs; life history, iden- 
tification, habits, suitability as pets, etc. Non-technical, but sound and broad. 130 
photos. 253pp. 22982-3 Pa. $3.00 


Op New York IN Eaniy PHoToGRAPHS, edited by Mary Black. Your only chance 
to see New York City as it was 1853-1906, through 196 wonderful photographs 
from N.Y. Historical Society. Great Blizzard, Lincoln's funeral procession, great 
buildings. 228pp. 9 x 12. 22907-6 Pa. $6.00 


THE AMERICAN REVOLUTION, A PictuRE SOURCEBOOK, John Grafton. Wonderful Bi- 
centennial picture source, with 411 illustrations (contemporary and 19th cen- 
tury) showing battles, personalities, maps, events, flags, posters, soldier's life, 
ships, etc. all captioned and explained. A wonderful browsing book, supplement 
to other historical reading. 160pp. 9 x 12. 23226-3 Pa. $4.00 


PERSONAL NARRATIVE OF A PILGRIMAGE TO AL-MADINAH AND Meccan, Richard Bur- 
ton. Great travel classic by remarkably colorful personality. Burton, disguised as 
a Moroccan, visited sacred shrines of Islam, narrowly escaping death. Wonderful 
observations of Islamic life, customs, personalities. 47 illustrations. Total of 
959pp. 21217-3, 21218-1 Pa., Two val. set $10.00 


INCIDENTS OF TRAVEL IN CENTRAL AMERICA, CHIAPAS, AND YUCATAN, John L. 
Stephens. Almost single-handed discovery of Maya culture; exploration of ruined 
cities, monuments, temples; customs of Indians. 115 drawings. 892pp. 

22404-X, 22405-8 Pa., Two vol. set $8.00 
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Mopean Cntess Stratecy, Ludek Pachman. The use of the queen, the active king, 
exchanges, pawn play, the center, weak squares, etc. Section on rook alone worth 
price of the book. Stress on the moderns. Often considered the most important 
book on strategy. 314pp. 20290-9 Pa. $3.50 


Cness StrateGy, Edward Lasker. One of half-dozen great theoretical works in 
chess, shows principles of action above and beyond moves. Acclaimed by 
Capablanca, Keres, etc. 282pp. USO 20528-2 Pa. $3.00 


Cuess Praxis, THe Praxis oF My System, Aron Nimzovich. Founder of hyper- 
modern chess explains his profound, influential theories that have dominated 
much of 20th century chess. 109 illustrative games. 369pp. 20296-8 Pa. $3.50 


How To Puay THE Citess OpeNtncs, Eugene Znosko-Borovsky. Clear, profound ex- 
aminations of just what each opening is intended to do and how opponent can 
counter. Many sample games, questions and answers. |47pp. 22795-2 Pa. $2.00 


Tue ART oF Cness CompinaTion, Eugene Znosko-Borovsky. Modern explanation 
of principles, varieties, techniques and ideas behind them, illustrated with many 
examples from great players. 212pp. 20583-5 Pa. $2.50 


ComBINATIONS: THE HEART OF CHESS, Irving Chernev. Step-by-step explanation of 
intricacies of combinative play. 356 combinations by Tarrasch, Botvinnik, Keres, 
Steinitz, Anderssen, Morphy, Marshall, Capablanca, others, all annotated. 245 
nies 21744-2 Pa. $3.00 


How to Pray Cuess Enpincs, Eugene Znosko-Borovsky. Thorough instruction 
manual by fine teacher analyzes each piece individually; many common endgame 
situations. Examines games by Steinitz, Alekhine, Lasker, others. Emphasis on 
understanding. 288pp. 21170-3 Pa. $2.75 


Morpny’s GaMEs OF Cuess, Philip W. Sergeant. Romantic history, 54 games of 
greatest player of all time against Anderssen, Bird, Paulsen, Harrwitz; 52 games 
at odds; 52 blindfold; 100 consultation, informal, other games. Analyses by An- 
derssen, Steinitz, Morphy himself. 352pp. 20386-7 Pa. $4.00 


500 Master Games oF Cuess, S. Tartakower, J. du Mont. Vast collection of great 
chess games from 1798-1938, with much material nowhere else readily available. 
Fully annotated, arranged by opening for easier study. 665pp. 23208-5 Pa. $6.00 


Tur Soviet SCHOOL OF Cress, Alexander Kotov and M. Yudovich. Authoritative 
work on modem Russian chess. History, conceptual background. 128 fully anno- 
tated games (most unavailable elsewhere) by Botvinnik, Keres, Smyslov, Tal, 
Petrosian, Spassky, more. 390pp. 20026-4 Pa. $3.95 


WONDERS AND Curiosities OF Cuess, Irving Chernev. A lifetime's accumulation of 
such wonders and curiosities as the longest won game, shortest game, chess 
problem with mate in 1220 moves, and much more unusual material —356 items 
in all, over 160 complete games. 146 diagrams. 203pp. 23007-4 Pa. $3.50 
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VISUAL ILLUSIONS: THEIR CAUSES, CHARACTERISTICS, AND APPLICATIONS, Mat- 
thew Luckiesh. Thorough description and discussion of optical illusion, geometric 
and perspective, particularly ; size and shape distortions, i}lusions of color, of motion ; 
natural illusions; use of illusion in art and magic, industry, etc. Most useful today 
with op art, also for classical art. Scores of effects illustrated. Introduction by 
William H. Ittleson. 100 illustrations. xxi +- 252pp. 

21530-X Paperbound $2.50 


A HANDBOOK OF ANATOMY FOR ART STUDENTS, Arthur Thomson. Thorough, vir- 
tually exhaustive coverage of skeletal structure, musculature, etc. Full text, supple- 
mented by anatomical diagrams and drawings and by photographs of undraped 
figures. Unique in its comparison of male and female forms, pointing out differences 
of contour, texture, form. 211 figures, 40 drawings, 86 photographs. xx +- 459pp. 
5% x 8%. 21163-0 Paperbound $5.00 


150 MASTERPIECES OF DRAWING, Selected by Anthony Toney. Full page reproduc- 
tions of drawings from the early 16th to the end of the 18th century, all beautifully 
reproduced: Rembrandt, Michelangelo, Diirer, Fragonard, Urs, Graf, Wouwerman, 
many others. First-rate browsing book, model book for artists. xviii + 150pp. 
83% x 11%. 21032-4 Paperbound’ $4.00 


THE LATER Work OF AUBREY BEARDSLEY, Aubrey Beardsley. Exotic, erotic, 
ironic masterpieces in full maturity: Comedy Ballet, Venus and Tannhauscr, Pierrot, 
Lysistrata, Rape of the Lock, Savoy material, Ali Baba, Volpone, etc. This material 
revolutionized the art world, and is still powerful, fresh, brilliant. With The Early 
Work, all Beardsley’s finest work. 174 plates, 2 in color. xiv + 176pp. 8% x 11. 

21817-1 Paperbound $4.00 


DRrAwINGs OF REMBRANDT, Rembrandt van Rijn. Complete reproduction of fabu- 
lously rare edition by Lippmann and Hofstede de Groot, completely ceedited, up- 
dated, improved by Prof. Seymour Slive, Fogg Museum. Portraits, Biblical sketches, 
landscapes, Oriental types, nudes, episodes from classical mythology—All Rem- 
brandt's fertile genius. Also selection of drawings by his pupils and fellowers. 
“Stunning volumes,” Saturday Review. 550 illustrations. Ixxviii + 552pp. 
9% x 12%. 21485-0, 21486-9 Two volumes, Paperbound $12.00 


THE DISASTERS OF War, Francisco Goya. One of the masterpieces of Western civi- 
lization—83 etchings that record Goya's shattering, bitter reaction to the Napoleonic 
war that swept through Spain after the insurrection of 1808 and to war in general. 
Reprint of the first edition, with three additional plates from Boston's Museum of 
Fine Arts. All plates facsimile size, Introduction by Philip Hofer, Fogg Museum. 
v + 97pp. 934 x 814. 21872-4 Paperbound $3.00 


Grapuic Works OF ODILON REDON. Largest collection of Redon's graphic works 
ever assembled: 172 lithographs, 28 etchings and engravings, 9 drawings. These 
include some of his most famous works, All the plates from Odilon Redon: oeuvre 
graphique complet, plus additional plates. New introduction and caption translations 
by Alfred Werner. 209 illustrations. xxvii -- 209pp. 94% x 124%. 

21966-8 Paperbound $6.00 
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SLEEPING Beauty, illustrated by Arthur Rackham. Perhaps the fullest, most 
delightful version ever, told by C.S. Evans. Rackham’s best work. 49 illustrations. 
110pp. 77s x 10%. 29756-1 Pa. $2.00 


THE WONDERFUL Wizanb oF Oz, L. Frank Baum. Facsimile in full color of Ameri- 
ca’s finest children’s classic. Introduction by Martin Gardner. 143 illustrations by 
W.W. Denslow. 267pp. 20691-2 Pa. $3.00 


Goors axp How To Be Tue, Gelett Burgess. Classic tongue-in-cheek mas- 
querading as etiquette book. 87 verses, 170 cartoons as Goops demonstrate vir- 
tues of table manners, neatness, courtesy, more. 88pp. 6'4 x 914. 

22233-0 Pa. $2.00 


Tue Brownies, THEIR Book, Palmer Cox. Small as mice, cunning as foxes, ex- 
uberant, mischievous, Brownies go to zoo, toy shop, seashore, circus, more. 24 
verse adventures. 266 illustrations. 144pp. 65/5 x 9'4. 21265-3 Pa. $2.50 


Bitty Waiskers: THE AUTOBIOGRAPHY OF A Goat, Frances Trego Montgomery. 
Escapades of that rambunctious goat. Favorite from turn of the century America. 
24 illustrations. 259pp. 22345-0 Pa. $2.75 


THE ROCKET Book, Peter Newell. Fritz, janitor’s kid, sets off rocket in basement of 
apartment house; an ingenious hole punched through every page traces course of 
rocket. 22 duotone drawings, verses. 48pp. 67 x 83/. 22044-3 Pa. $1.50 


Peck’s Bap Boy anxp His Pa, George W. Peck. Complete double-volume of great 
American childhood classic. Hennery’s ingenious pranks against outraged pom- 
posity of pa and the grocery man. 97 illustrations. Introduction by E.F. Bleiler. 
347pp. 20497-9 Pa. $2.50 


Tue TALE oF PeTen Rabat, Beatrix Potter. The inimitable Peter's terrifying ad- 
venture in Mr. McGregor’s garden, with all 27 wonderful, full-color Potter il- 
lustrations. S5pp. 414 x 544. USO 22827-4 Pa. $1.00 


THE TALE OF Mas. TiGGy-WinkLe, Beatrix Potter. Your child will love this story 
about a very special hedgehog and all 27 wonderful, full-color Potter illustra- 
tions. 57pp. 44 x S'é. USO 20546-0 Pa. $1.00 


THe TALE oF BENJAMIN BuNNy, Beatrix Potter. Peter Rabbit’s cousin coaxes him 
back into Mr. McGregor’s garden for a whole new set of adventures. A favorite 
with children. All 27 full-color illustrations. 59pp. 44 x 54. 

USO 21102-9 Pa. $1.00 


THE Merry ADVENTURES OF RoBIN Hoop, Howard Pyle. Facsimile of original 
(1883) edition, finest modern version of English outlaw’s adventures. 23 illustra- 
tions by Pyle. 296pp. 64 x 9'4. 22043-5 Pa. $4.00 


Two LITTLE SavaGes, Ernest Thompson Seton. Adventures of two boys who lived 
as Indians; explaining Indian ways, woodlore, pioneer methods. 293 illustrations. 
286pp. 20985-7 Pa. $3.00 
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THE JourNAL oF Henny D. THoreau, edited by Bradford Torrey, F.H. Allen. Com- 
pe reprinting of 14 volumes, 1837-1861, over two million words; the source- 
ooks for Walden, etc. Definitive. All original sketches, plus 75 photographs. In- 

troduction by Walter Harding. Total of 1804pp. 81% x 12'4. 
20312-3, 20313-1 Clothbd., Two vol. set $50.00 


MASTERS OF THE Drama, John Gassner. Most comprehensive history of the drama, 
every tradition from Greeks to modern Europe and America, including Orient. 
Covers 800 dramatists, 2000 plays; biography, plot summaries, criticism, theatre 
history, etc. 77 illustrations. 890pp. 20100-7 Clothbd. $10.00 


Guost AnD Horror Stories OF AMBROSE BIERCE, Ambrose Bierce. 23 modern hor- 
ror stories: The Eyes of the Panther, The Damned Thing, ete., plus the dream- 
essay Visions of the Night. Edited by E.F. Bleiler. 199pp. | 20767-6 Pa. $2.00 


Best Guost Storirs, Algernon Blackwood. 13 great stories by foremost British 
20th century supermaturalist. The Willows, The Wendigo, Ancient Sorceries, 
others. Edited by E.F. Bleiler. 366pp. USO 22977-7 Pa. $3.00 


Tue Best TaLes oF HOFFMANN, E.T.A. Hoffmann. 10 of Hoffmann’s most impor- 
tant stories, in modern re-editings of standard translations: Nutcracker and the 
King of Mice, The Golden Flowerpot, etc. 7 illustrations by Hoffmann. Edited by 
EF. Bleiler. 458pp. 21793-0 Pa. $3.95 


Best Guost Stones oF J.S. LeFanu, J. Sheridan LeFanu. 16 stories by greatest 
Victorian master: Green Tea, Carmilla, Haunted Baronet, The Familiar, etc. 
Mostly unavailable elsewhere. Edited by EF. Bleiler. 8 illustrations. 467pp. 

20415-4 Pa. $4.00 


SupPERNATURAL Honnon IN LITERATURE, H.P. Lovecraft. Great modern American 
supernaturalist brilliantly surveys history of genre to 1930's, summarizing, 
evaluating scores of hooks. Necessary for every student, lover of form. Introduc- 
tion by ELF. Bleiler. 111pp. 20105-8 Pa. $1.50 


THREE GoTHic NoveELs, ed. by E.F. Bleiler. Full texts Castle of Otranto, Walpole; 
Vathek, Beckford; The Vampyre, Polidori; Fragment of a Novel, Lord Byron. 
33\pp. 21232-7 Pa. $3.00 


SEVEN SCIENCE Fiction Nove ts, H.G. Wells. Full novels. First Men in the Moon, 

Island of Dr. Moreau, War of the Worlds, Food of the Gods, Invisible Man, Time 

Machine, In the Days of the Comet. A basic science-fiction library. 1015pp. 
USO 20264-X Clothbd. $6.00 


Lapy AUDLEY’s SECRET, Mary E. Braddon. Great Victorian mystery classic, 
beautifully plotted, suspenseful; praised by Thackeray, Boucher, Starrett, others. 
What happened to beautiful, vicious Lady Audley’s husband? Introduction by 
Norman Donaldson. 286pp. 23011-2 Pa. $3.00 
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Tue ART Deco Stv-e, ed. by Thecdore Menten. Furniture, jewelry, metalwork, 
ceramics, fabrics, lighting fixtures, interior decors, exteriors, graphics from pure 
French sources. Best sampling around. Over 400 photographs. 183pp. 838 x 11%. 

22824-X Pa. $4.00 


THE GENTLEMAN AND CABINET Maken’s Director, Thomas Chippendale. Full 
reprint, 1762 style book, most influential of all time; chairs, tables, sofas, mirrors, 
cabinets, etc. 200 plates, plus 24 photographs of surviving pieces. 249pp. 978 x 
12%. 21601-2 Pa. $6.00 


Pine FuaNITURE OF Ean.y New ENGLAND, Russell H. Kettell. Basic book. Thorough 
historical text, plus 200 illustrations of boxes, highboys, candlesticks, desks, etc. 
47 7pp. Tis x 10%. 20145-7 Clothbd. $12.50 


Ontentat. Rucs, ANTIQUE AND Mopers, Walter A. Hawley. Persia, Turkey, 
Caucasus, Central Asia, China, other traditions. Best general survey of all 
aspects: styles and periods, manufacture, uses, symbols and their interpretation, 
and identification. 96 illustrations, 11 in color. 320pp. 61/6 x 9'4. 

22366-3 Pa. $5.00 


DECORATIVE ANTIQUE IRONWORK, Henry R. d’Allemagne. Photographs of 4500 
iron artifacts from world's finest collection, Rouen. Hinges, locks, candelabra, 
weapons, lighting devices, clocks, tools, from Roman times to mid-19th century. 
Nothing else comparable to it. 420pp. 9 x 12. 22082-6 Pa. $8.50 


THE ComMPLETE Book OF DoLL MAKING AND COLLECTING, Catherine Christopher. 
Instructions, patterns for dozens of dolls, from rag doll on up to elaborate, 
historically accurate figures. Mould faces, sew clothing, make doll houses, etc. 
Also collecting information. Many illustrations. 288pp. 6 x 9. 22066-4 Pa. $3.00 


ANTIQUE Papen Dotts: 1915-1920, edited by Arnold Arnold. 7 antique cut-out 
dolls and 24 costumes from 1915-1920, selected by Arnold Arnold from his col- 
lection of rare children’s books and entertainments, all in full color. 32pp. 9% x 
12%. 23176-3 Pa. $2.00 


ANTIQUE PaFER DoLts: Tite EpwarbiaNn Eaa, Epinal. Full-color reproductions of 
two historic series of paper dolls that show clothing styles in 1908 and at the 
beginning of the First World War. 8 two-sided, stand-up dolls and 32 complete, 
two-sided costumes. Full instructions for assembling included. 32pp. 9% x 12%. 

23175-5 Pa. $2.00 


A History of Costume, Carl Kthler, Emma von Sichardt. Egypt, Babylon, Greece 
up through 19th century Europe; based on surviving pieces, art works, etc. Full 
text and 595 illustrations, including many clear, measured patterns for reproduc- 
ing historic costume. Practical. 464pp. 21030-8 Pa. $4.00 


Ear.y AMERICAN Locomotives, John H. White, Jr. Finest locomotive engravings 
from late 19th century: historical (1804-1874), main-line (after 1870), special, 
foreign, etc. 147 plates. 200pp. 115% x 814. 22772-3 Pa. $3.50 
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Tie Macic Movin Picturt& Book, Bliss, Sands & Co. The pictures in this book 
move! Volcanoes erupt, a house bums, a serpentine dancer wiggles her way 
through a number. By using a specially ruled acetate screen provided, you can 
obtain these and 15 other startling effects. Originally “The Motograph Moving 
Picture Book.” 32pp. 8% x 11. 23224-7 Pa. $1.75 


STRING FicuREs AND How To Make Tre, Caroline F. Jayne. Fullest, clearest in- 
structions on string figures from around world: Eskimo, Navajo, Lapp, Europe, 
more. Cats cradle, moving spear, lightning, stars. Introduction by A.C. Haddon. 
950 illustrations. 407pp. 20152-X Pa. $3.50 


PAPER FOLDING FoR Becinners, William D. Murray and Francis J. Rigney. Clearest 
book on market for making origami sail boats, roosters, frogs that move legs, cups, 
bonbon boxes. 40 projects. More than 275 illustrations. Photographs. 94pp. 

20713-7 Pa. $1.25 


INDIAN SIGN LancuaceE, William Tomkins. Over 525 signs developed by Sioux, 
Blackfoot, Cheyenne, Arapahoe and other tribes. Written instructions and 
diagrams: how to make words, construct sentences. Also 290 pictographs of Sioux 
and Ojibway tribes. 11] 1 pp. 6! x 9". 22029-X Pa. $1.50 


Boomenancs: How To MAKE AND THROW THEM, Bernard S. Mason. Easy to make 
and throw, dozens of designs: cross-stick, pinwheel, boomabird, tumblestick, 
Australian curved stick boomerang. Complete throwing instructions. All safe. 
99pp. 23028-7 Pa. $1.75 


25 Kites Tat FLy, Leslie Hunt. Full, easy to follow instructions for kites made 
from inexpensive materials. Many novelties. Reeling, raising, designing your own. 
70 illustrations. 110pp. 22550-X Pa. $1.25 


TRICKS AND GAMES ON THE PooL TABLE, Fred Herrmann. 79 tricks and games, 
some solitaires, some for 2 or more players, some competitive; mystifying shots 
and throws, unusual carom, tricks involving cork, coins, a hat, more. 77 figures. 
95pp. 21814-7 Pa. $1.25 


Woopcrart AND CaMPING, Bernard S. Mason. How to make a quick emergency 
shelter, select woods that will burn immediately, make do with limited supplies, 
etc. Also making many things out of wood, rawhide, hark, at camp. Formerly 
titled Woodcraft. 295 illustrations. 580pp. 21951-8 Pa. $4.00 


AN INTRODUCTION TO CHESS Moves AND Tactics SimPLy EXPLAINED, Leonard 
Barden. Informal intermediate introduction: reasons for moves, tactics, openings, 
traps, positional play, endgame. Isolates patterns. 102pp. USO 21210-6 Pa. $1.35 


Lasker's MANUAL OF Cuess, Dr. Emanuel Lasker. Great world champion offers 
very thorough coverage of all aspects of chess. Combinations, position play, open- 
ings, endgame, aesthetics of chess, philosophy of struggle, much more. Filled with 
analyzed games. 390pp. 20640-8 Pa. $4.00 


CATALOGUE OF DOVER BOOKS 


Tue Rep Fairy Book, Andrew Lang. Lang's color fairy books have long been 

children’s favorites, This volume includes Rapunzel, Jack and the Bean-stalk and 

35 other stories, familiar and unfamiliar. 4 plates, 93 illustrations x + 367pp. 
21673-X Paperbound $3.00 


THE Bue Fairy Book, Andrew Lang. Lang's tales come from all countries and all 
times. Here are 37 tales from Grimm, the Arabian Nights, Greek Mythology, and 
other fascinating sources. 8 plates, 130 illustrations. xi + 390pp. 

21437-0 Paperbound $3.50 


HOUSEHOLD STORIES BY THE BROTHERS GRIMM. Classic English-language edition 
of the well-known tales — Rumpelstiltskin, Sow White, Hansel and Gretel, The 
Twelve Brothers, Faithful John, Rapunzel, Tom Thumb (52 stories in all). Trans- 
lated into simple, straightforward English by Lucy Crane. Ornamented with head- 
pieces, vignettes, elaborate decorative initials and a dozen full-page illustrations by 
Walter Crane. x ++ 269pp. 21080-4 Paperbound $3.00 


THE Merry ADVENTURES OF ROBIN Hoop, Howard Pyle. The finest modern ver- 
sions of the traditional ballads and tales about the great English outlaw. Howard 
Pyle’s complete prose version, with every word, every illustration of the first edition. 
Do not confuse this facsimile of the original (1883) with modern editions that 
change text or illustrations. 23 plates plus many page decorations. xxii +- 296pp. 

22043-5 Paperbound $4.00 


Tue Story oF KING ARTHUR AND His KNIGHTS, Howard Pyle. The finest chil- 
dren's version of the life of King Arthur; brilliantly retold by Pyle, with 48 of his 
most imaginative illustrations. xviii ++ 313pp. 6% x 914. 

21445-1 Paperbound $3.50 


THE WONDERFUL WizARD OF Oz, L. Frank Baum. America’s finest children’s 
book in facsimile of first edition with all Denslow illustrations in full color. The 
edition a child should have. Introduction by Martin Gardner. 23 color plates, 
scores of drawings. iv +- 267pp. 20691-2 Paperbound $3.00 


THE MARVELOUS LAND OF Oz, L. Frank Baum. The second Oz book, every bit as 
imaginative as the Wizard. The hero is a boy named Tip, but the Scarecrow and the 
Tin Woodman are back, as is the Oz magic. 16 color plates, 120 drawings by John 
R. Neill. 287pp. 20692-0 Paperbound $3.00 


THE MAGICAL MONARCH OF Mo, L. Frank Baum. Remarkable adventures in 2 land 
even stranger than Oz. The best of Baum’s books not in the Oz series. 15 color 
plates and dozens of drawings by Frank Verbeck. xviii +- 237pp. 

21892-9 Paperbound $2.95 


THE Bap CuitLp’s Book oF Beasts, MorE BEASTS FOR WoRSE CHILDREN, A 
MoRAL ALPHABET, Hilaire Belloc. Three complete humor classics in one volume. 
Be kind to the frog, and do not call him names . . . and 28 other whimsical animals. 
Familiar favorites and some not so well known. Illustrated by Basil Blackwell. 
156pp. (USO) 20749-8 Paperbound $2.00 
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150 MASTERPIECES OF Drawinc, edited by Anthony Toney. 150 plates, early 15th 
century to end of 18th century; Rembrandt, Michelangelo, Diirer, Fragonard, 
Watteau, Wouwerman, many others. 150pp. 89/8 x 11’. 21032-4 Pa. $4.00 


Tite GOLDEN AGE OF THE Poster, Hayward and Blanche Cirker. 70 extraordinary 
posters in full colors, from Mautres de l’Affiche, Mucha, Lautrec, Bradley, Cheret, 
Beardsley, many others. 93/8 x 12°4. 22753-7 Pa. $4.95 

21718-3 Clothbd. $7.95 


SIMPLICIssiMUS, selection, translations and text by Stanley Appelbaum. 180 satiri- 
cal drawings, 16 in full color, from the famous German weekly magazine in the 
years 1896 to 1926. 24 artists included: Grosz, Kley, Pascin, Kubin, Kollwitz, plus 
Heine, Thény, Bruno Paul, others. 172pp. 8'4 x 12%. 23098-8 Pa. $5.00 

23099-6 Clothbd. $10.00 


Tite Eanty Work OF AUBREY BEARDSLEY, Aubrey Beardsley. 157 plates, 2 in color: 
Manon Lescaut, Madame Bovary, Morte d’Arthur, Salome, other. Introduction by 
H. Marillier. 175pp. 8 x 11. 21816-3 Pa. $4.00 


THE LaTeR Work OF AUBREY BEARDSLEY, Aubrey Beardsley. Exotic masterpieces 
of full maturity: Venus and Tannhauser, Lysistrata, Rape of the Lock, Volpone, 
Savoy material, etc. 174 plates, 2 in color. 176pp. 8'4 x 11. 21817-1 Pa. $4.00 


DRawinGs OF WILLIAM BLAKE, William Blake. 92 plates from Book of Job, Divine 
Comedy, Paradise Lost, visionary heads, mythological figures, Laocodn, etc. 
Selection, introduction, commentary by Sir Geoffrey Keynes. 178pp. 8% x 11. 

22303-5 Pa. $3.50 


Lonpon: A PILGRIMAGE, Gustave Dore, Blanchard Jerrold. Squalor, riches, mis- 
ery, beauty of mid-Victorian metropolis; 55 wonderful plates, 125 other illustra- 
tions, full social, cultural text by Jerrold. 191pp. of text. 8 x 11. 

22306-X Pa. $5.00 


Tne ComPLETE Woopcuts OF ALBRECHT Duren, edited by Dr. W. Kurth. 346 in 
all: Old Testament, St. Jerome, Passion, Life of Virgin, Apocalypse, many others. 
Introduction by Campbell Dodgson. 285pp. 8'4 x 12%. 21097-9 Pa. $6.00 


Tne Disasters of War, Francisco Goya. 83 etchings record horrors of Napoleonic 
wars in Spain and war in general. Reprint of Ist edition, plus 3 additional plates. 
Introduction by Philip Hofer. 97pp. 955 x 814. 21872-4 Pa. $3.00 


ENGRAVINGS OF Hocartn, William Hogarth. 101 of Hogarth’s greatest works: 
Rake’s Progress, Harlot’s Progress, Illustrations for Hudibras, Midnight Modern 
Conversation, Before and After, Beer Street and Gin Lane, many more. Full com- 
mentary. 256pp. 11 x 14. 22479-1 Pa. $7.00 

23023-6 Clothbd. $13.50 


Primitive Ant, Franz Boas. Great anthropologist on ceramics, textiles, wood, 
stone, metal, etc.; patterns, technology, symbols, styles. All areas, but fullest on 
Northwest Coast Indians. 350 illustrations. 378pp. 20025-6 Pa. $3.75 
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Ecyretias Macic, E.A. Wallis Budge. Foremost Egyptologist, curator at British 
Museum, on charms, curses, amulets, doll magic, transformations, contro! of 
demons, deific appearances, feats of great magicians. Many texts cited. 19 il- 
lustrations. 234pp. USO 22681-6 Pa. $2.50 


Tne Leypen Papyrus: AX Ecyptias Macican Book, edited by F. LI. Griffith, Her- 
bert Thompson. Egyptian sorcerer’s manual contains scores of spells: sex magic of 
various sorts, occult information, evoking visions, removing evil magic, ete. 
Transliteration faces translation. 207 pp. 22994-7 Pa. $2.50 


THE MALLEUS MALEFICARUM OF KRAMER AND SPRENGER, translated, edited by Mon- 
tague Summers. Full text of most important witchhunter’s “Bible,” used by both 
Catholics and Protestants. Theory of witches, manifestations, remedies, etc. In- 
dispensable to serious student. 278pp. 6°/ x 10. USO 22802-9 Pa. $3.95 


Lost Continents, L. Sprague de Camp. Great science-fiction author, finest, 
fullest study: Atlantis, Lemuria, Mu, Hyperborea, etc. Lost Tribes, Irish in pre- 
Columbian America, root races; in history, literature, art, occultism. Necessary to 
everyone concerned with theme. 17 illustrations. 348pp. 22668-9 Pa. $3.50 


Tie Compete Books oF Cuarces Fort, Charles Fort. Book of the Damned, Lo!, 
Wild Talents, New Lands. Greatest compilation of data: celestial appearances, 
flying saucers, falls of frogs, strange disappearances, inexplicable data not recog- 
nized by science. Inexhaustible, painstakingly documented. Do not confuse with 
modern charlatanry. Introduction by Damon Knight. Total of 1126pp. 

23094-5 Clothbd. $15.00 


Faps AND FALLACIES IN THE NAME OF SCIENCE, Martin Gardner. Fair, witty ap- 
praisal of cranks and quacks of science: Atlantis, Lemuria, flat earth, Velikovsky, 
orgone energy, Bridey Murphy, medical fads, ete. 373pp. —- 20394-8 Pa. $3.50 


Hoaxes, Curtis D. MacDougall. Unbelievably rich account of great hoaxes: 
Locke's moon hoax, Shakespearean forgeries, Loch Ness monster, Disumbra- 
tionist school of art, dozens more; also psychology of hoaxing. 54 illustrations. 
338pp. 20465-0 Pa. $3.50 


Tite GENTLE ART OF MaKING ENEMIES, James A.M. Whistler. Greatest wit of his day 
deflates Wilde, Ruskin, Swinburne; strikes back at inane critics, exhibitions. 
Highly readable classic of impressionist revolution hy great painter. Introduction 
by Alfred Werner. 334pp. 21875-9 Pa. $4.00 


Tue Book or Tea, Kakuzo Okakura. Minor classic of the Orient: entertaining, 
charming explanation, interpretation of traditional Japanese culture in terms of 
tea ceremony. Edited by E.F. Bleiler. Total of 94pp. 20070-) Pa. $1.25 


Prices subject te change without notice. 

Available at your book dealer or write for free catalogue to Dept. Gl, Dover 
Publications, Inc., 180 Varick St.. N.Y., N.Y. 10084. Dover publishes more than 
150 hooks each year on science, elementary and advanced mathematics, biology, 
music, art, literary history, social sciences and other areas. 


